2024-01-28T12:45:22.316 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-28T12:45:22.316 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799 --name matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} --owner scheduled_matan@teuthology -- /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/orig.config.yaml 2024-01-28T12:45:22.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T12:45:22.429 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799 branch: wip-matanb-crimson-pg-map-logs description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: null first_in_suite: false job_id: '7535799' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-pg-map-logs ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson osd obc lru size: 10 crimson seastar smp: 3 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 00c0d54d57237e78360214c1fefe250a2610dcfe ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 00c0d54d57237e78360214c1fefe250a2610dcfe workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 7eb9e33f53ab64f23bcf0a2a31509f3442110e4e owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 7326 sha1: 00c0d54d57237e78360214c1fefe250a2610dcfe sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 7eb9e33f53ab64f23bcf0a2a31509f3442110e4e targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+e/mN47WL+k2UShx4+QdYe2PoCv9kFTOWTk6NUp62n3KES5c+ZwdMP9U4M1mNve/sZ1c2dU0QAIgqEU58p7p8= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIydCY7OmjmfsVwUzUdkkPVrOiO0zWzLJGxQNS0ZorlF1m7YkqFZ0KHhg6Emxon2MVNwuKOiZXxfyftCEFTDDvA= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c856e0f9244dd29a50697123b480d63f16a4ed81 timestamp: 2024-01-28_12:38:29 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-28T12:45:22.429 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa; will attempt to use it 2024-01-28T12:45:22.430 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks 2024-01-28T12:45:22.430 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-28T12:45:22.431 INFO:teuthology.task.internal:Checking packages... 2024-01-28T12:45:22.457 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '00c0d54d57237e78360214c1fefe250a2610dcfe' 2024-01-28T12:45:22.457 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-28T12:45:22.457 INFO:teuthology.packaging:ref: None 2024-01-28T12:45:22.457 INFO:teuthology.packaging:tag: None 2024-01-28T12:45:22.457 INFO:teuthology.packaging:branch: wip-matanb-crimson-pg-map-logs 2024-01-28T12:45:22.457 INFO:teuthology.packaging:sha1: 00c0d54d57237e78360214c1fefe250a2610dcfe 2024-01-28T12:45:22.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-pg-map-logs 2024-01-28T12:45:22.602 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-933.g00c0d54d 2024-01-28T12:45:22.603 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-28T12:45:22.610 INFO:teuthology.task.internal:no buildpackages task found 2024-01-28T12:45:22.610 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-28T12:45:22.644 INFO:teuthology.task.internal:Saving configuration 2024-01-28T12:45:22.656 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-28T12:45:22.677 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-28T12:45:22.703 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-28 12:40:23.044797', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+e/mN47WL+k2UShx4+QdYe2PoCv9kFTOWTk6NUp62n3KES5c+ZwdMP9U4M1mNve/sZ1c2dU0QAIgqEU58p7p8='} 2024-01-28T12:45:22.723 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-28 12:40:23.046374', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIydCY7OmjmfsVwUzUdkkPVrOiO0zWzLJGxQNS0ZorlF1m7YkqFZ0KHhg6Emxon2MVNwuKOiZXxfyftCEFTDDvA='} 2024-01-28T12:45:22.723 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-28T12:45:22.733 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-01-28T12:45:22.733 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-01-28T12:45:22.733 INFO:teuthology.run_tasks:Running task console_log... 2024-01-28T12:45:22.827 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d17b02ee0>, signals=[15]) 2024-01-28T12:45:22.828 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-28T12:45:22.834 INFO:teuthology.task.internal:Opening connections... 2024-01-28T12:45:22.834 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-01-28T12:45:22.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:45:22.981 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-01-28T12:45:22.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:45:23.067 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-28T12:45:23.075 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-01-28T12:45:23.094 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-01-28T12:45:23.094 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-01-28T12:45:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-28T12:45:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-01-28T12:45:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T12:45:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-28T12:45:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-28T12:45:23.153 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-01-28T12:45:23.173 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-01-28T12:45:23.190 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-01-28T12:45:23.191 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-01-28T12:45:23.248 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-01-28T12:45:23.248 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-28T12:45:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-28T12:45:23.249 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-01-28T12:45:23.273 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-28T12:45:23.282 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-28T12:45:23.329 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-28T12:45:23.330 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-01-28T12:45:23.332 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-01-28T12:45:23.348 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-28T12:45:23.361 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-28T12:45:23.361 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-01-28T12:45:23.390 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-01-28T12:45:23.436 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-28T12:45:23.523 INFO:teuthology.run_tasks:Running task kernel... 2024-01-28T12:45:23.538 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-01-28T12:45:23.538 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-01-28T12:45:23.538 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-28T12:45:23.539 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-28T12:45:23.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T12:45:23.557 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-01-28T12:45:23.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T12:45:23.558 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-01-28T12:45:23.614 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-361.el9.x86_64 2024-01-28T12:45:23.614 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.14.0-361.el9.x86_64 2024-01-28T12:45:23.614 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-01-28T12:45:23.615 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-361.el9.x86_64 2024-01-28T12:45:23.616 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-361.el9.x86_64 2024-01-28T12:45:23.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-01-28T12:45:25.666 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2024-01-28T12:45:25.732 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2024-01-28T12:45:27.150 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-01-28T12:45:27.182 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-01-28T12:45:28.815 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-01-28T12:45:29.557 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-01-28T12:45:32.199 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 19 MB 00:02 2024-01-28T12:45:33.516 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-01-28T12:45:37.983 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 5.9 MB 00:02 2024-01-28T12:45:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 5.9 MB 00:02 2024-01-28T12:45:39.742 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 15 kB 00:00 2024-01-28T12:45:40.642 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 15 kB 00:00 2024-01-28T12:45:41.593 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-01-28T12:45:41.893 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2024-01-28T12:45:46.526 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-01-28T12:45:46.868 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-01-28T12:45:48.808 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-28T12:45:48.808 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-28T12:45:48.902 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:45:48.913 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:45:48.913 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:45:48.913 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:45:48.913 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:45:48.913 INFO:teuthology.orchestra.run.smithi137.stdout: kernel x86_64 5.14.0-411.el9 CentOS-BaseOS 6.0 M 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core x86_64 5.14.0-411.el9 CentOS-BaseOS 21 M 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules x86_64 5.14.0-411.el9 CentOS-BaseOS 39 M 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core x86_64 5.14.0-411.el9 CentOS-BaseOS 33 M 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-01-28T12:45:48.914 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:45:48.915 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 99 M 2024-01-28T12:45:48.915 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 122 M 2024-01-28T12:45:48.915 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:45:49.208 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-28T12:45:49.208 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-28T12:45:49.298 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:45:49.307 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-411.el9 CentOS-BaseOS 6.0 M 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-411.el9 CentOS-BaseOS 21 M 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-411.el9 CentOS-BaseOS 39 M 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-411.el9 CentOS-BaseOS 33 M 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-01-28T12:45:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:45:49.309 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 99 M 2024-01-28T12:45:49.309 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 122 M 2024-01-28T12:45:49.309 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:45:49.327 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): kernel-5.14.0-411.el9.x86_64.rpm 15 MB/s | 6.0 MB 00:00 2024-01-28T12:45:49.994 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): kernel-core-5.14.0-411.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-01-28T12:45:50.227 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-411.el9.x86_64.rpm 6.5 MB/s | 6.0 MB 00:00 2024-01-28T12:45:51.027 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-411.el9.x86_64.rpm 12 MB/s | 21 MB 00:01 2024-01-28T12:45:52.036 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): kernel-modules-core-5.14.0-411.el9.x86_6 12 MB/s | 33 MB 00:02 2024-01-28T12:45:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-5.14.0-411.el9.x86_64.rpm 14 MB/s | 39 MB 00:02 2024-01-28T12:45:52.353 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-core-5.14.0-411.el9.x86_6 16 MB/s | 33 MB 00:02 2024-01-28T12:45:52.356 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:45:52.356 INFO:teuthology.orchestra.run.smithi031.stdout:Total 33 MB/s | 99 MB 00:03 2024-01-28T12:45:52.357 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:45:52.586 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): kernel-modules-5.14.0-411.el9.x86_64.rpm 11 MB/s | 39 MB 00:03 2024-01-28T12:45:52.589 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:45:52.589 INFO:teuthology.orchestra.run.smithi137.stdout:Total 27 MB/s | 99 MB 00:03 2024-01-28T12:45:52.590 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:45:52.784 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:45:52.784 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:45:52.955 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:45:52.955 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:45:53.851 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:45:53.851 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:45:54.012 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:45:54.013 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:45:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:45:56.424 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:45:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-411.el9.x86_64 1/4 2024-01-28T12:45:57.096 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-core-5.14.0-411.el9.x86_64 1/4 2024-01-28T12:45:57.228 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-28T12:45:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-28T12:45:59.990 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-28T12:46:00.171 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-28T12:46:00.246 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-28T12:46:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-28T12:46:04.864 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-28T12:46:04.983 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-28T12:46:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-core-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:38.859 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:38.859 INFO:teuthology.orchestra.run.smithi137.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-28T12:46:38.860 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:46:38.931 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:39.316 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:39.316 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-5.14.0-411.el9.x86_64 1/4 2024-01-28T12:46:39.316 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-28T12:46:39.316 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-core-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-5.14.0-411.el9.x86_64 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core-5.14.0-411.el9.x86_64 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-5.14.0-411.el9.x86_64 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core-5.14.0-411.el9.x86_64 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:46:39.747 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:46:40.267 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-01-28T12:46:40.395 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-411.el9.x86_64 2024-01-28T12:46:40.396 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-411.el9.x86_64 2024-01-28T12:46:40.396 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-411.el9.x86_64 2024-01-28T12:46:40.396 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-28T12:46:40.397 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-28T12:46:40.397 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-01-28T12:46:40.397 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-01-28T12:46:40.397 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-01-28T12:46:40.503 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-411.el9.x86_64 2024-01-28T12:46:40.504 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-411.el9.x86_64 2024-01-28T12:46:40.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.06 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 68.el9 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Group : Unspecified 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 8264254 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-28T12:46:40.542 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : builder@centos.org 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:46:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-01-28T12:46:40.544 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-28T12:46:40.544 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 5.14.0-411.el9.x86_64 2024-01-28T12:46:40.544 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-28T12:46:41.217 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-01-28T12:46:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:41.737 INFO:teuthology.orchestra.run.smithi031.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-28T12:46:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:46:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-411.el9.x86_64 1/4 2024-01-28T12:46:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-28T12:46:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-28T12:46:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-28T12:46:42.664 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-01-28T12:46:42.667 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-01-28T12:46:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-411.el9.x86_64 4/4 2024-01-28T12:46:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:46:42.672 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:46:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-411.el9.x86_64 2024-01-28T12:46:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-411.el9.x86_64 2024-01-28T12:46:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-411.el9.x86_64 2024-01-28T12:46:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-411.el9.x86_64 2024-01-28T12:46:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:46:42.672 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:46:42.722 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.E7qRff1T7b 2024-01-28T12:46:42.722 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.E7qRff1T7b 2024-01-28T12:46:42.790 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.E7qRff1T7b 2024-01-28T12:46:42.926 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.E7qRff1T7b is 6KB 2024-01-28T12:46:42.946 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.E7qRff1T7b 2024-01-28T12:46:42.960 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-01-28T12:46:43.025 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-28T12:46:43.025 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-28T12:46:43.025 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-28T12:46:43.025 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-411.el9.x86_64.conf 2024-01-28T12:46:43.026 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-411.el9.x86_64 2024-01-28T12:46:43.157 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-01-28T12:46:43.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-01-28T12:46:43.291 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-411.el9.x86_64 2024-01-28T12:46:43.292 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-411.el9.x86_64 2024-01-28T12:46:43.292 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-411.el9.x86_64 2024-01-28T12:46:43.292 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-28T12:46:43.292 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-28T12:46:43.292 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-01-28T12:46:43.292 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-01-28T12:46:43.293 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-01-28T12:46:43.405 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-411.el9.x86_64 2024-01-28T12:46:43.405 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-411.el9.x86_64 2024-01-28T12:46:43.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-01-28T12:46:43.443 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-01-28T12:46:43.443 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-01-28T12:46:43.443 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-01-28T12:46:43.443 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 68.el9 2024-01-28T12:46:43.443 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8264254 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-01-28T12:46:43.444 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:46:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-01-28T12:46:43.446 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-28T12:46:43.446 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-411.el9.x86_64 2024-01-28T12:46:43.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-28T12:46:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-01-28T12:46:45.375 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-28T12:46:45.410 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-01-28T12:46:45.412 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-01-28T12:46:45.427 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.lmteMdWCqZ 2024-01-28T12:46:45.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lmteMdWCqZ 2024-01-28T12:46:45.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.lmteMdWCqZ 2024-01-28T12:46:45.647 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.lmteMdWCqZ is 6KB 2024-01-28T12:46:45.696 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.lmteMdWCqZ 2024-01-28T12:46:45.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-01-28T12:46:45.779 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-28T12:46:45.779 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-28T12:46:45.779 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-28T12:46:45.779 INFO:teuthology.orchestra.run.smithi031.stdout:15949e3c88704300882482eddfd7baae-5.14.0-411.el9.x86_64.conf 2024-01-28T12:46:45.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-411.el9.x86_64 2024-01-28T12:46:45.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-01-28T12:47:13.212 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-01-28T12:47:13.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-28T12:47:13.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:47:15.928 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-01-28T12:47:15.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-28T12:47:15.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:47:31.644 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-01-28T12:47:34.460 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-01-28T12:47:40.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-28T12:47:40.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:47:43.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-28T12:47:43.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:47:43.708 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-01-28T12:47:55.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-28T12:47:55.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:48:01.852 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-01-28T12:48:13.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-28T12:48:13.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:48:14.269 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-01-28T12:48:14.756 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-28T12:48:14.756 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-411.el9.x86_64"... 2024-01-28T12:48:14.757 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-01-28T12:48:14.770 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-411.el9.x86_64 2024-01-28T12:48:14.770 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-411.el9.x86_64 vs 5.14.0-411.el9.x86_64 2024-01-28T12:48:14.770 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-28T12:48:14.771 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-28T12:48:15.772 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-01-28T12:48:15.772 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-28T12:48:15.874 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-01-28T12:48:15.908 DEBUG:teuthology.parallel:result is None 2024-01-28T12:48:27.551 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-01-28T12:48:27.975 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-28T12:48:27.976 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-411.el9.x86_64"... 2024-01-28T12:48:27.976 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-01-28T12:48:27.990 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-411.el9.x86_64 2024-01-28T12:48:27.990 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-411.el9.x86_64 vs 5.14.0-411.el9.x86_64 2024-01-28T12:48:27.990 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-28T12:48:27.990 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-28T12:48:28.991 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-01-28T12:48:28.991 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-28T12:48:29.080 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-01-28T12:48:29.124 DEBUG:teuthology.parallel:result is None 2024-01-28T12:48:29.125 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-28T12:48:29.133 INFO:teuthology.task.internal:Creating test directory... 2024-01-28T12:48:29.134 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-28T12:48:29.137 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-28T12:48:29.159 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-28T12:48:29.182 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-28T12:48:29.265 INFO:teuthology.task.internal:Creating archive directory... 2024-01-28T12:48:29.265 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-28T12:48:29.268 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-28T12:48:29.300 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-28T12:48:29.314 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-28T12:48:29.315 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-28T12:48:29.337 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-28T12:48:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-28T12:48:29.372 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-28T12:48:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-28T12:48:29.385 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-28T12:48:29.387 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-28T12:48:29.394 INFO:teuthology.task.internal:Configuring sudo... 2024-01-28T12:48:29.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-28T12:48:29.426 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-28T12:48:29.455 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-28T12:48:29.468 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-28T12:48:29.468 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-28T12:48:29.498 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-28T12:48:29.512 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-28T12:48:29.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-28T12:48:29.647 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-28T12:48:29.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-28T12:48:29.774 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:48:29.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-28T12:48:29.842 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-28T12:48:29.883 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-28T12:48:30.000 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-28T12:48:30.042 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-28T12:48:30.085 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:48:30.085 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-28T12:48:30.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-01-28T12:48:30.154 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-01-28T12:48:30.210 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-28T12:48:30.232 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-28T12:48:30.727 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-28T12:48:30.735 INFO:teuthology.task.internal:Starting timer... 2024-01-28T12:48:30.736 INFO:teuthology.run_tasks:Running task pcp... 2024-01-28T12:48:30.774 INFO:teuthology.run_tasks:Running task selinux... 2024-01-28T12:48:30.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-01-28T12:48:30.893 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-01-28T12:48:30.895 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-01-28T12:48:30.955 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-01-28T12:48:30.957 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-28T12:48:30.957 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-01-28T12:48:30.992 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-01-28T12:48:30.992 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-01-28T12:48:31.019 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-01-28T12:48:31.019 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2024-01-28T12:48:31.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-28T12:48:31.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T12:48:31.060 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-28T12:48:31.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T12:48:31.088 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-28T12:48:31.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-01-28T12:48:31.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-01-28T12:48:31.171 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-28T12:48:31.181 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-28T12:48:31.201 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-28T12:48:31.202 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-28T12:52:36.658 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2024-01-28T12:52:36.659 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-28T12:52:36.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:52:36.755 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-01-28T12:52:36.839 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-01-28T12:52:36.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-28T12:52:36.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-28T12:52:36.927 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-01-28T12:52:37.003 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-01-28T12:52:37.004 INFO:teuthology.run_tasks:Running task clock... 2024-01-28T12:52:37.014 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-28T12:52:37.014 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-28T12:52:37.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-28T12:52:37.017 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-28T12:52:37.018 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-28T12:52:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-28T12:52:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-28T12:52:37.074 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-28T12:52:37.091 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-28T12:52:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-01-28T12:52:37.109 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-01-28T12:52:37.120 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-01-28T12:52:37.126 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-28T12:52:37.133 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-01-28T12:52:37.142 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-28T12:52:37.150 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-28T12:52:37.166 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-28T12:52:37.197 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-01-28T12:52:37.201 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-28T12:52:37.201 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-01-28T12:52:37.201 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.201 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.201 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.201 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.217 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-01-28T12:52:37.220 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-28T12:52:37.220 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-01-28T12:52:37.220 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.221 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.221 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.221 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T12:52:37.222 INFO:teuthology.run_tasks:Running task install... 2024-01-28T12:52:37.231 DEBUG:teuthology.task.install:project ceph 2024-01-28T12:52:37.231 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe'}} 2024-01-28T12:52:37.232 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe'} 2024-01-28T12:52:37.232 INFO:teuthology.task.install:Using flavor: crimson 2024-01-28T12:52:37.238 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-28T12:52:37.238 INFO:teuthology.task.install:extra packages: [] 2024-01-28T12:52:37.238 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe', 'tag': None, 'wait_for_package': False} 2024-01-28T12:52:37.239 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=00c0d54d57237e78360214c1fefe250a2610dcfe 2024-01-28T12:52:37.240 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe', 'tag': None, 'wait_for_package': False} 2024-01-28T12:52:37.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=00c0d54d57237e78360214c1fefe250a2610dcfe 2024-01-28T12:52:37.385 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/ 2024-01-28T12:52:37.386 INFO:teuthology.task.install.rpm:Package version is 19.0.0-933.g00c0d54d 2024-01-28T12:52:37.389 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/ 2024-01-28T12:52:37.389 INFO:teuthology.task.install.rpm:Package version is 19.0.0-933.g00c0d54d 2024-01-28T12:52:37.505 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-28T12:52:37.506 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:52:37.506 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-28T12:52:37.534 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-01-28T12:52:37.534 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/00c0d54d57237e78360214c1fefe250a2610dcfe/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-28T12:52:37.556 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-pg-map-logs/00c0d54d57237e78360214c1fefe250a2610dcfe/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-28T12:52:37.556 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:52:37.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-28T12:52:37.584 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-01-28T12:52:37.584 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/00c0d54d57237e78360214c1fefe250a2610dcfe/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-28T12:52:37.601 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-28T12:52:37.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-28T12:52:37.680 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-28T12:52:37.732 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-28T12:52:37.746 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-01-28T12:52:37.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-01-28T12:52:37.799 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-01-28T12:52:37.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-01-28T12:52:37.989 INFO:teuthology.orchestra.run.smithi137.stdout:74 files removed 2024-01-28T12:52:38.016 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw 2024-01-28T12:52:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:74 files removed 2024-01-28T12:52:38.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-01-28T12:52:38.712 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 221 kB/s | 83 kB 00:00 2024-01-28T12:52:38.861 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-01-28T12:52:39.070 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 31 kB/s | 9.5 kB 00:00 2024-01-28T12:52:39.237 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 29 kB/s | 9.5 kB 00:00 2024-01-28T12:52:39.563 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-01-28T12:52:39.570 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-01-28T12:52:40.115 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2024-01-28T12:52:40.390 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 8.1 MB/s | 6.4 MB 00:00 2024-01-28T12:52:41.584 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-01-28T12:52:41.732 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-01-28T12:52:42.631 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-01-28T12:52:43.300 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-01-28T12:52:44.474 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-01-28T12:52:46.542 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-01-28T12:52:49.241 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 5.9 MB 00:01 2024-01-28T12:52:51.075 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-01-28T12:52:51.810 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 5.9 MB 00:01 2024-01-28T12:52:52.171 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2024-01-28T12:52:53.194 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 15 kB 00:00 2024-01-28T12:52:54.071 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 28 MB/s | 20 MB 00:00 2024-01-28T12:52:57.235 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 9.5 kB/s | 1.7 kB 00:00 2024-01-28T12:52:59.288 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-01-28T12:52:59.863 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:52:59.875 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:52:59.875 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:52:59.875 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:52:59.875 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:52:59.875 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 9.0 M 2024-01-28T12:52:59.875 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-01-28T12:52:59.875 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 3.1 M 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 3.1 M 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 5.1 M 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 18 M 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 25 k 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-28T12:52:59.876 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 752 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 497 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 4.8 M 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 45 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 130 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 162 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-28T12:52:59.877 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 321 k 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 298 k 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 99 k 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout:Install 27 Packages 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 2 Packages 2024-01-28T12:52:59.878 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:52:59.879 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 54 M 2024-01-28T12:52:59.879 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:53:00.778 INFO:teuthology.orchestra.run.smithi137.stdout:(1/29): ceph-radosgw-19.0.0-933.g00c0d54d.el9.x 11 MB/s | 9.0 MB 00:00 2024-01-28T12:53:00.828 INFO:teuthology.orchestra.run.smithi137.stdout:(2/29): ceph-selinux-19.0.0-933.g00c0d54d.el9.x 505 kB/s | 25 kB 00:00 2024-01-28T12:53:00.937 INFO:teuthology.orchestra.run.smithi137.stdout:(3/29): libcephfs2-19.0.0-933.g00c0d54d.el9.x86 6.8 MB/s | 752 kB 00:00 2024-01-28T12:53:01.037 INFO:teuthology.orchestra.run.smithi137.stdout:(4/29): ceph-base-19.0.0-933.g00c0d54d.el9.x86_ 4.7 MB/s | 5.1 MB 00:01 2024-01-28T12:53:01.062 INFO:teuthology.orchestra.run.smithi137.stdout:(5/29): libradosstriper1-19.0.0-933.g00c0d54d.e 3.9 MB/s | 497 kB 00:00 2024-01-28T12:53:01.146 INFO:teuthology.orchestra.run.smithi137.stdout:(6/29): python3-ceph-argparse-19.0.0-933.g00c0d 541 kB/s | 45 kB 00:00 2024-01-28T12:53:01.246 INFO:teuthology.orchestra.run.smithi137.stdout:(7/29): python3-ceph-common-19.0.0-933.g00c0d54 1.3 MB/s | 130 kB 00:00 2024-01-28T12:53:01.338 INFO:teuthology.orchestra.run.smithi137.stdout:(8/29): python3-cephfs-19.0.0-933.g00c0d54d.el9 1.7 MB/s | 162 kB 00:00 2024-01-28T12:53:01.439 INFO:teuthology.orchestra.run.smithi137.stdout:(9/29): python3-rados-19.0.0-933.g00c0d54d.el9. 3.1 MB/s | 321 kB 00:00 2024-01-28T12:53:01.522 INFO:teuthology.orchestra.run.smithi137.stdout:(10/29): python3-rbd-19.0.0-933.g00c0d54d.el9.x 3.5 MB/s | 298 kB 00:00 2024-01-28T12:53:01.631 INFO:teuthology.orchestra.run.smithi137.stdout:(11/29): librgw2-19.0.0-933.g00c0d54d.el9.x86_6 8.1 MB/s | 4.8 MB 00:00 2024-01-28T12:53:01.656 INFO:teuthology.orchestra.run.smithi137.stdout:(12/29): python3-rgw-19.0.0-933.g00c0d54d.el9.x 744 kB/s | 99 kB 00:00 2024-01-28T12:53:01.773 INFO:teuthology.orchestra.run.smithi137.stdout:(13/29): boost-program-options-1.75.0-8.el9.x86 746 kB/s | 106 kB 00:00 2024-01-28T12:53:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:(14/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-01-28T12:53:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:(15/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-01-28T12:53:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:(16/29): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-01-28T12:53:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:(17/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-01-28T12:53:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:(18/29): python3-prettytable-0.7.2-27.el9.noarc 826 kB/s | 41 kB 00:00 2024-01-28T12:53:02.042 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:53:02.052 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:53:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:53:02.052 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:53:02.052 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:53:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 9.0 M 2024-01-28T12:53:02.052 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-01-28T12:53:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 3.1 M 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 3.1 M 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 5.1 M 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 18 M 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 25 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 752 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-28T12:53:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 497 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 4.8 M 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 45 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 130 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 162 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 321 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 298 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 99 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-28T12:53:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout:Install 27 Packages 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 54 M 2024-01-28T12:53:02.055 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:53:02.108 INFO:teuthology.orchestra.run.smithi137.stdout:(19/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.5 MB/s | 309 kB 00:00 2024-01-28T12:53:02.158 INFO:teuthology.orchestra.run.smithi137.stdout:(20/29): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-01-28T12:53:02.242 INFO:teuthology.orchestra.run.smithi137.stdout:(21/29): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-01-28T12:53:02.267 INFO:teuthology.orchestra.run.smithi137.stdout:(22/29): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-01-28T12:53:02.292 INFO:teuthology.orchestra.run.smithi137.stdout:(23/29): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-01-28T12:53:02.351 INFO:teuthology.orchestra.run.smithi137.stdout:(24/29): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-01-28T12:53:02.376 INFO:teuthology.orchestra.run.smithi137.stdout:(25/29): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-01-28T12:53:02.435 INFO:teuthology.orchestra.run.smithi137.stdout:(26/29): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-01-28T12:53:03.060 INFO:teuthology.orchestra.run.smithi137.stdout:(27/29): ceph-common-19.0.0-933.g00c0d54d.el9.x 5.8 MB/s | 18 MB 00:03 2024-01-28T12:53:03.119 INFO:teuthology.orchestra.run.smithi137.stdout:(28/29): librbd1-19.0.0-933.g00c0d54d.el9.x86_6 4.5 MB/s | 3.1 MB 00:00 2024-01-28T12:53:03.177 INFO:teuthology.orchestra.run.smithi137.stdout:(29/29): librados2-19.0.0-933.g00c0d54d.el9.x86 3.9 MB/s | 3.1 MB 00:00 2024-01-28T12:53:03.181 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:53:03.181 INFO:teuthology.orchestra.run.smithi137.stdout:Total 16 MB/s | 54 MB 00:03 2024-01-28T12:53:03.182 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:53:03.213 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:53:03.214 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:53:03.689 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:53:03.689 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:53:04.147 INFO:teuthology.orchestra.run.smithi031.stdout:(1/29): ceph-base-19.0.0-933.g00c0d54d.el9.x86_ 2.7 MB/s | 5.1 MB 00:01 2024-01-28T12:53:04.180 INFO:teuthology.orchestra.run.smithi031.stdout:(2/29): ceph-selinux-19.0.0-933.g00c0d54d.el9.x 759 kB/s | 25 kB 00:00 2024-01-28T12:53:04.384 INFO:teuthology.orchestra.run.smithi031.stdout:(3/29): libcephfs2-19.0.0-933.g00c0d54d.el9.x86 3.7 MB/s | 752 kB 00:00 2024-01-28T12:53:04.514 INFO:teuthology.orchestra.run.smithi031.stdout:(4/29): libradosstriper1-19.0.0-933.g00c0d54d.e 3.6 MB/s | 497 kB 00:00 2024-01-28T12:53:04.615 INFO:teuthology.orchestra.run.smithi031.stdout:(5/29): ceph-radosgw-19.0.0-933.g00c0d54d.el9.x 3.8 MB/s | 9.0 MB 00:02 2024-01-28T12:53:04.665 INFO:teuthology.orchestra.run.smithi031.stdout:(6/29): python3-ceph-argparse-19.0.0-933.g00c0d 909 kB/s | 45 kB 00:00 2024-01-28T12:53:04.715 INFO:teuthology.orchestra.run.smithi031.stdout:(7/29): python3-ceph-common-19.0.0-933.g00c0d54 2.5 MB/s | 130 kB 00:00 2024-01-28T12:53:04.782 INFO:teuthology.orchestra.run.smithi031.stdout:(8/29): python3-cephfs-19.0.0-933.g00c0d54d.el9 2.4 MB/s | 162 kB 00:00 2024-01-28T12:53:04.891 INFO:teuthology.orchestra.run.smithi031.stdout:(9/29): python3-rados-19.0.0-933.g00c0d54d.el9. 2.9 MB/s | 321 kB 00:00 2024-01-28T12:53:04.983 INFO:teuthology.orchestra.run.smithi031.stdout:(10/29): python3-rbd-19.0.0-933.g00c0d54d.el9.x 3.2 MB/s | 298 kB 00:00 2024-01-28T12:53:05.033 INFO:teuthology.orchestra.run.smithi031.stdout:(11/29): python3-rgw-19.0.0-933.g00c0d54d.el9.x 1.9 MB/s | 99 kB 00:00 2024-01-28T12:53:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:53:05.142 INFO:teuthology.orchestra.run.smithi031.stdout:(12/29): boost-program-options-1.75.0-8.el9.x86 979 kB/s | 106 kB 00:00 2024-01-28T12:53:05.192 INFO:teuthology.orchestra.run.smithi031.stdout:(13/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-01-28T12:53:05.234 INFO:teuthology.orchestra.run.smithi031.stdout:(14/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-01-28T12:53:05.292 INFO:teuthology.orchestra.run.smithi031.stdout:(15/29): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-01-28T12:53:05.334 INFO:teuthology.orchestra.run.smithi031.stdout:(16/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-01-28T12:53:05.376 INFO:teuthology.orchestra.run.smithi031.stdout:(17/29): python3-prettytable-0.7.2-27.el9.noarc 995 kB/s | 41 kB 00:00 2024-01-28T12:53:05.465 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2024-01-28T12:53:05.504 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 2/31 2024-01-28T12:53:05.593 INFO:teuthology.orchestra.run.smithi031.stdout:(18/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.4 MB/s | 309 kB 00:00 2024-01-28T12:53:05.595 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 2/31 2024-01-28T12:53:05.686 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 3/31 2024-01-28T12:53:05.706 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 4/31 2024-01-28T12:53:05.710 INFO:teuthology.orchestra.run.smithi031.stdout:(19/29): librgw2-19.0.0-933.g00c0d54d.el9.x86_6 4.0 MB/s | 4.8 MB 00:01 2024-01-28T12:53:05.798 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 4/31 2024-01-28T12:53:05.827 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/31 2024-01-28T12:53:05.852 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/31 2024-01-28T12:53:05.869 INFO:teuthology.orchestra.run.smithi031.stdout:(20/29): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-01-28T12:53:05.889 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 7/31 2024-01-28T12:53:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:(21/29): libarrow-doc-9.0.0-10.el9.noarch.rpm 141 kB/s | 26 kB 00:00 2024-01-28T12:53:05.933 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-933.g00c0d54d.el9.x 8/31 2024-01-28T12:53:05.945 INFO:teuthology.orchestra.run.smithi031.stdout:(22/29): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-01-28T12:53:05.995 INFO:teuthology.orchestra.run.smithi031.stdout:(23/29): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-01-28T12:53:06.003 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 9/31 2024-01-28T12:53:06.025 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 10/31 2024-01-28T12:53:06.037 INFO:teuthology.orchestra.run.smithi031.stdout:(24/29): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-01-28T12:53:06.096 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 10/31 2024-01-28T12:53:06.104 INFO:teuthology.orchestra.run.smithi031.stdout:(25/29): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-01-28T12:53:06.146 INFO:teuthology.orchestra.run.smithi031.stdout:(26/29): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-01-28T12:53:06.162 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 11/31 2024-01-28T12:53:06.266 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2024-01-28T12:53:06.300 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/31 2024-01-28T12:53:06.340 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2024-01-28T12:53:06.373 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2024-01-28T12:53:06.411 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/31 2024-01-28T12:53:06.444 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2024-01-28T12:53:06.672 INFO:teuthology.orchestra.run.smithi031.stdout:(27/29): ceph-common-19.0.0-933.g00c0d54d.el9.x 4.1 MB/s | 18 MB 00:04 2024-01-28T12:53:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2024-01-28T12:53:06.772 INFO:teuthology.orchestra.run.smithi031.stdout:(28/29): librados2-19.0.0-933.g00c0d54d.el9.x86 4.7 MB/s | 3.1 MB 00:00 2024-01-28T12:53:06.798 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 19/31 2024-01-28T12:53:06.830 INFO:teuthology.orchestra.run.smithi031.stdout:(29/29): librbd1-19.0.0-933.g00c0d54d.el9.x86_6 4.5 MB/s | 3.1 MB 00:00 2024-01-28T12:53:06.834 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:53:06.834 INFO:teuthology.orchestra.run.smithi031.stdout:Total 11 MB/s | 54 MB 00:04 2024-01-28T12:53:06.834 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:53:06.876 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:53:06.876 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:53:06.897 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 19/31 2024-01-28T12:53:06.928 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 20/31 2024-01-28T12:53:06.964 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2024-01-28T12:53:07.356 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:53:07.356 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:53:07.421 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 22/31 2024-01-28T12:53:07.451 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 23/31 2024-01-28T12:53:07.505 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 23/31 2024-01-28T12:53:07.582 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 24/31 2024-01-28T12:53:07.620 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-933.g00c0d54d.el9.x86 25/31 2024-01-28T12:53:08.753 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:53:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2024-01-28T12:53:09.216 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 2/31 2024-01-28T12:53:09.308 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 2/31 2024-01-28T12:53:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 3/31 2024-01-28T12:53:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 4/31 2024-01-28T12:53:09.533 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 4/31 2024-01-28T12:53:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/31 2024-01-28T12:53:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/31 2024-01-28T12:53:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 7/31 2024-01-28T12:53:09.679 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.0.0-933.g00c0d54d.el9.x 8/31 2024-01-28T12:53:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 26/31 2024-01-28T12:53:09.751 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 9/31 2024-01-28T12:53:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 26/31 2024-01-28T12:53:09.779 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 10/31 2024-01-28T12:53:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 10/31 2024-01-28T12:53:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 11/31 2024-01-28T12:53:09.998 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2024-01-28T12:53:10.029 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/31 2024-01-28T12:53:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2024-01-28T12:53:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2024-01-28T12:53:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/31 2024-01-28T12:53:10.172 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2024-01-28T12:53:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 26/31 2024-01-28T12:53:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 27/31 2024-01-28T12:53:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 27/31 2024-01-28T12:53:10.469 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-28T12:53:10.469 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-28T12:53:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:10.513 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 28/31 2024-01-28T12:53:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2024-01-28T12:53:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 19/31 2024-01-28T12:53:10.650 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 19/31 2024-01-28T12:53:10.682 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 20/31 2024-01-28T12:53:10.718 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2024-01-28T12:53:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 22/31 2024-01-28T12:53:11.230 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 23/31 2024-01-28T12:53:11.285 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 23/31 2024-01-28T12:53:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 24/31 2024-01-28T12:53:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.0.0-933.g00c0d54d.el9.x86 25/31 2024-01-28T12:53:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 26/31 2024-01-28T12:53:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 26/31 2024-01-28T12:53:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 26/31 2024-01-28T12:53:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 27/31 2024-01-28T12:53:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 27/31 2024-01-28T12:53:14.484 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-28T12:53:14.484 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-28T12:53:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 28/31 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 28/31 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-01-28T12:53:19.898 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:20.791 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 29/31 2024-01-28T12:53:20.814 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 29/31 2024-01-28T12:53:20.814 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:53:20.814 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-28T12:53:20.815 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-28T12:53:20.815 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-28T12:53:20.815 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:20.816 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-01-28T12:53:20.892 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-01-28T12:53:20.893 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2024-01-28T12:53:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 1/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 2/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 3/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 4/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 5/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 6/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 7/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-933.g00c0d54d.el9.x 8/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-933.g00c0d54d.el9.x86 9/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 10/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 11/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 12/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 13/31 2024-01-28T12:53:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 21/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 22/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 25/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2024-01-28T12:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2024-01-28T12:53:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 28/31 2024-01-28T12:53:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2024-01-28T12:53:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 30/31 2024-01-28T12:53:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2024-01-28T12:53:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:23.137 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-01-28T12:53:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-28T12:53:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-28T12:53:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:23.140 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:53:23.522 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-test 2024-01-28T12:53:24.095 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:27 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:53:24.258 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 121 M 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout: cryptopp x86_64 8.6.0-1.el9 epel 1.3 M 2024-01-28T12:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: libasan x86_64 11.4.1-3.el9 CentOS-AppStream 417 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 158 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: libubsan x86_64 11.4.1-3.el9 CentOS-AppStream 186 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout:Install 10 Packages 2024-01-28T12:53:24.260 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:24.261 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 123 M 2024-01-28T12:53:24.261 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 1.1 G 2024-01-28T12:53:24.261 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:53:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 28/31 2024-01-28T12:53:24.545 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-01-28T12:53:24.545 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-01-28T12:53:24.546 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-01-28T12:53:24.546 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-01-28T12:53:24.546 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-01-28T12:53:24.546 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-01-28T12:53:24.546 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-01-28T12:53:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:24.579 INFO:teuthology.orchestra.run.smithi137.stdout:(1/10): libasan-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 417 kB 00:00 2024-01-28T12:53:24.713 INFO:teuthology.orchestra.run.smithi137.stdout:(2/10): libubsan-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 186 kB 00:00 2024-01-28T12:53:24.738 INFO:teuthology.orchestra.run.smithi137.stdout:(3/10): libcephsqlite-19.0.0-933.g00c0d54d.el9. 372 kB/s | 158 kB 00:00 2024-01-28T12:53:24.763 INFO:teuthology.orchestra.run.smithi137.stdout:(4/10): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-01-28T12:53:24.788 INFO:teuthology.orchestra.run.smithi137.stdout:(5/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-01-28T12:53:24.814 INFO:teuthology.orchestra.run.smithi137.stdout:(6/10): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-01-28T12:53:24.839 INFO:teuthology.orchestra.run.smithi137.stdout:(7/10): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-01-28T12:53:24.923 INFO:teuthology.orchestra.run.smithi137.stdout:(8/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.4 MB/s | 122 kB 00:00 2024-01-28T12:53:24.981 INFO:teuthology.orchestra.run.smithi137.stdout:(9/10): cryptopp-8.6.0-1.el9.x86_64.rpm 7.5 MB/s | 1.3 MB 00:00 2024-01-28T12:53:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 29/31 2024-01-28T12:53:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 29/31 2024-01-28T12:53:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:53:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-28T12:53:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-28T12:53:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-28T12:53:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-01-28T12:53:25.577 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-01-28T12:53:25.577 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 1/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 2/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 3/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 4/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 5/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 6/31 2024-01-28T12:53:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 7/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.0.0-933.g00c0d54d.el9.x 8/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.0.0-933.g00c0d54d.el9.x86 9/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 10/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 11/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 12/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 13/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2024-01-28T12:53:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 21/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 22/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 25/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 28/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2024-01-28T12:53:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 30/31 2024-01-28T12:53:27.766 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2024-01-28T12:53:27.766 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:27.766 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-01-28T12:53:27.766 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.766 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.767 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-3.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-28T12:53:27.768 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:27.769 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:53:28.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-01-28T12:53:28.643 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:29 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:53:28.804 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:53:28.805 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:53:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:53:28.805 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:53:28.805 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:53:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 121 M 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: cryptopp x86_64 8.6.0-1.el9 epel 1.3 M 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: libasan x86_64 11.4.1-3.el9 CentOS-AppStream 417 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 158 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: libubsan x86_64 11.4.1-3.el9 CentOS-AppStream 186 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:53:28.806 INFO:teuthology.orchestra.run.smithi031.stdout:Install 10 Packages 2024-01-28T12:53:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:53:28.807 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 123 M 2024-01-28T12:53:28.807 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.1 G 2024-01-28T12:53:28.807 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:53:29.125 INFO:teuthology.orchestra.run.smithi031.stdout:(1/10): libasan-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 417 kB 00:00 2024-01-28T12:53:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:(2/10): libubsan-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 186 kB 00:00 2024-01-28T12:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:(3/10): socat-1.7.4.1-5.el9.x86_64.rpm 2.3 MB/s | 300 kB 00:00 2024-01-28T12:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:(4/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-01-28T12:53:29.476 INFO:teuthology.orchestra.run.smithi031.stdout:(5/10): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2024-01-28T12:53:29.535 INFO:teuthology.orchestra.run.smithi031.stdout:(6/10): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-01-28T12:53:29.660 INFO:teuthology.orchestra.run.smithi031.stdout:(7/10): libcephsqlite-19.0.0-933.g00c0d54d.el9. 205 kB/s | 158 kB 00:00 2024-01-28T12:53:30.194 INFO:teuthology.orchestra.run.smithi031.stdout:(8/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 228 kB/s | 122 kB 00:00 2024-01-28T12:53:30.352 INFO:teuthology.orchestra.run.smithi031.stdout:(9/10): cryptopp-8.6.0-1.el9.x86_64.rpm 1.5 MB/s | 1.3 MB 00:00 2024-01-28T12:53:30.393 INFO:teuthology.orchestra.run.smithi137.stdout:(10/10): ceph-test-19.0.0-933.g00c0d54d.el9.x86 20 MB/s | 121 MB 00:06 2024-01-28T12:53:30.395 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:53:30.395 INFO:teuthology.orchestra.run.smithi137.stdout:Total 20 MB/s | 123 MB 00:06 2024-01-28T12:53:30.396 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:53:30.415 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:53:30.415 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:53:31.267 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:53:31.267 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:53:32.406 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:53:32.497 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2024-01-28T12:53:32.535 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cryptopp-8.6.0-1.el9.x86_64 2/10 2024-01-28T12:53:32.575 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/10 2024-01-28T12:53:32.606 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-16.el9.x86_64 4/10 2024-01-28T12:53:32.651 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2024-01-28T12:53:32.679 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/10 2024-01-28T12:53:32.708 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 7/10 2024-01-28T12:53:32.749 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libasan-11.4.1-3.el9.x86_64 8/10 2024-01-28T12:53:32.778 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 9/10 2024-01-28T12:53:44.959 INFO:teuthology.orchestra.run.smithi031.stdout:(10/10): ceph-test-19.0.0-933.g00c0d54d.el9.x86 7.5 MB/s | 121 MB 00:16 2024-01-28T12:53:44.964 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:53:44.974 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.6 MB/s | 123 MB 00:16 2024-01-28T12:53:44.975 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:53:44.988 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:53:44.989 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:53:45.853 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:53:45.853 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:53:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:53:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2024-01-28T12:53:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cryptopp-8.6.0-1.el9.x86_64 2/10 2024-01-28T12:53:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/10 2024-01-28T12:53:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.6-16.el9.x86_64 4/10 2024-01-28T12:53:47.313 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2024-01-28T12:53:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/10 2024-01-28T12:53:47.370 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 7/10 2024-01-28T12:53:47.412 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libasan-11.4.1-3.el9.x86_64 8/10 2024-01-28T12:53:47.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 9/10 2024-01-28T12:53:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 9/10 2024-01-28T12:53:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 10/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 10/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 1/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 2/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libasan-11.4.1-3.el9.x86_64 3/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 4/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-16.el9.x86_64 7/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 8/10 2024-01-28T12:53:57.680 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cryptopp-8.6.0-1.el9.x86_64 9/10 2024-01-28T12:53:58.138 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2024-01-28T12:53:58.138 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:58.138 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:53:58.138 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 cryptopp-8.6.0-1.el9.x86_64 2024-01-28T12:53:58.138 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-16.el9.x86_64 libasan-11.4.1-3.el9.x86_64 2024-01-28T12:53:58.139 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 libubsan-11.4.1-3.el9.x86_64 2024-01-28T12:53:58.139 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.9.6-1.el9.5.x86_64 socat-1.7.4.1-5.el9.x86_64 2024-01-28T12:53:58.139 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 yaml-cpp-0.6.3-5.el9.x86_64 2024-01-28T12:53:58.139 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:58.139 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:53:58.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph 2024-01-28T12:53:59.087 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:02 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:53:59.248 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:53:59.250 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================== 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================== 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 6.5 k 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 2.0 M 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 1.5 M 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 245 k 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 4.8 M 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 117 M 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-28T12:53:59.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-28T12:53:59.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================== 2024-01-28T12:53:59.253 INFO:teuthology.orchestra.run.smithi137.stdout:Install 36 Packages 2024-01-28T12:53:59.254 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:53:59.254 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 130 M 2024-01-28T12:53:59.254 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 1.4 G 2024-01-28T12:53:59.254 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:53:59.871 INFO:teuthology.orchestra.run.smithi137.stdout:(1/36): ceph-19.0.0-933.g00c0d54d.el9.x86_64.rp 29 kB/s | 6.5 kB 00:00 2024-01-28T12:53:59.988 INFO:teuthology.orchestra.run.smithi137.stdout:(2/36): ceph-mds-19.0.0-933.g00c0d54d.el9.x86_6 5.9 MB/s | 2.0 MB 00:00 2024-01-28T12:54:00.339 INFO:teuthology.orchestra.run.smithi137.stdout:(3/36): ceph-mgr-19.0.0-933.g00c0d54d.el9.x86_6 2.1 MB/s | 1.5 MB 00:00 2024-01-28T12:54:00.455 INFO:teuthology.orchestra.run.smithi137.stdout:(4/36): ceph-mgr-modules-core-19.0.0-933.g00c0d 2.1 MB/s | 245 kB 00:00 2024-01-28T12:54:00.589 INFO:teuthology.orchestra.run.smithi137.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-01-28T12:54:00.631 INFO:teuthology.orchestra.run.smithi137.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-01-28T12:54:00.656 INFO:teuthology.orchestra.run.smithi137.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-01-28T12:54:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-01-28T12:54:00.723 INFO:teuthology.orchestra.run.smithi137.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-01-28T12:54:00.765 INFO:teuthology.orchestra.run.smithi137.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-01-28T12:54:00.841 INFO:teuthology.orchestra.run.smithi137.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-01-28T12:54:00.883 INFO:teuthology.orchestra.run.smithi137.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2024-01-28T12:54:00.916 INFO:teuthology.orchestra.run.smithi137.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.7 MB/s | 124 kB 00:00 2024-01-28T12:54:00.950 INFO:teuthology.orchestra.run.smithi137.stdout:(14/36): python3-requests-2.25.1-7.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2024-01-28T12:54:00.992 INFO:teuthology.orchestra.run.smithi137.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-01-28T12:54:01.067 INFO:teuthology.orchestra.run.smithi137.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-01-28T12:54:01.618 INFO:teuthology.orchestra.run.smithi137.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 275 kB/s | 151 kB 00:00 2024-01-28T12:54:01.751 INFO:teuthology.orchestra.run.smithi137.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 326 kB/s | 43 kB 00:00 2024-01-28T12:54:01.945 INFO:teuthology.orchestra.run.smithi137.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 887 kB/s | 172 kB 00:00 2024-01-28T12:54:02.168 INFO:teuthology.orchestra.run.smithi137.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-01-28T12:54:02.269 INFO:teuthology.orchestra.run.smithi137.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-01-28T12:54:02.377 INFO:teuthology.orchestra.run.smithi137.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-01-28T12:54:02.494 INFO:teuthology.orchestra.run.smithi137.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 198 kB/s | 23 kB 00:00 2024-01-28T12:54:02.620 INFO:teuthology.orchestra.run.smithi137.stdout:(24/36): ceph-mon-19.0.0-933.g00c0d54d.el9.x86_ 1.8 MB/s | 4.8 MB 00:02 2024-01-28T12:54:02.645 INFO:teuthology.orchestra.run.smithi137.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 129 kB/s | 19 kB 00:00 2024-01-28T12:54:02.772 INFO:teuthology.orchestra.run.smithi137.stdout:(26/36): python3-logutils-0.3.5-21.el9.noarch.r 370 kB/s | 46 kB 00:00 2024-01-28T12:54:02.896 INFO:teuthology.orchestra.run.smithi137.stdout:(27/36): python3-more-itertools-8.12.0-2.el9.no 631 kB/s | 79 kB 00:00 2024-01-28T12:54:02.963 INFO:teuthology.orchestra.run.smithi137.stdout:(28/36): python3-jaraco-text-3.2.0-6.el9.noarch 57 kB/s | 20 kB 00:00 2024-01-28T12:54:03.038 INFO:teuthology.orchestra.run.smithi137.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-01-28T12:54:03.089 INFO:teuthology.orchestra.run.smithi137.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 131 kB/s | 16 kB 00:00 2024-01-28T12:54:03.140 INFO:teuthology.orchestra.run.smithi137.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 896 kB/s | 90 kB 00:00 2024-01-28T12:54:03.248 INFO:teuthology.orchestra.run.smithi137.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 225 kB/s | 36 kB 00:00 2024-01-28T12:54:03.281 INFO:teuthology.orchestra.run.smithi137.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-01-28T12:54:03.392 INFO:teuthology.orchestra.run.smithi137.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-01-28T12:54:03.615 INFO:teuthology.orchestra.run.smithi137.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.1 MB/s | 427 kB 00:00 2024-01-28T12:54:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 9/10 2024-01-28T12:54:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 10/10 2024-01-28T12:54:05.365 INFO:teuthology.orchestra.run.smithi137.stdout:(36/36): ceph-osd-19.0.0-933.g00c0d54d.el9.x86_ 22 MB/s | 117 MB 00:05 2024-01-28T12:54:05.371 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:54:05.371 INFO:teuthology.orchestra.run.smithi137.stdout:Total 21 MB/s | 130 MB 00:06 2024-01-28T12:54:05.424 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:54:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:54:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:54:06.488 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:54:06.488 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:54:08.390 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:54:08.459 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-01-28T12:54:08.492 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-01-28T12:54:08.520 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-01-28T12:54:08.590 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-01-28T12:54:08.635 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-01-28T12:54:08.668 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-01-28T12:54:08.759 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-01-28T12:54:08.831 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-01-28T12:54:08.887 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-01-28T12:54:08.958 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-01-28T12:54:09.013 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-01-28T12:54:09.060 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 12/36 2024-01-28T12:54:09.108 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-01-28T12:54:09.164 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-01-28T12:54:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-01-28T12:54:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-01-28T12:54:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-01-28T12:54:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-01-28T12:54:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-01-28T12:54:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-01-28T12:54:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-01-28T12:54:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-01-28T12:54:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-01-28T12:54:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-01-28T12:54:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-01-28T12:54:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-01-28T12:54:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-01-28T12:54:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-01-28T12:54:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-01-28T12:54:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-01-28T12:54:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-933.g00c0d54d.el9.n 31/36 2024-01-28T12:54:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 32/36 2024-01-28T12:54:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 32/36 2024-01-28T12:54:10.457 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:10.457 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-28T12:54:10.457 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-28T12:54:10.457 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-28T12:54:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 10/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 1/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 2/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libasan-11.4.1-3.el9.x86_64 3/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 4/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.6-16.el9.x86_64 7/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 8/10 2024-01-28T12:54:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cryptopp-8.6.0-1.el9.x86_64 9/10 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.0.0-933.g00c0d54d.el9.x86_64 cryptopp-8.6.0-1.el9.x86_64 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.6-16.el9.x86_64 libasan-11.4.1-3.el9.x86_64 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.0.0-933.g00c0d54d.el9.x86_64 libubsan-11.4.1-3.el9.x86_64 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.9.6-1.el9.5.x86_64 socat-1.7.4.1-5.el9.x86_64 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 yaml-cpp-0.6.3-5.el9.x86_64 2024-01-28T12:54:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:13.796 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:54:14.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-01-28T12:54:14.720 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:15 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:54:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:54:14.887 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================== 2024-01-28T12:54:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:54:14.887 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================== 2024-01-28T12:54:14.887 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:54:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 6.5 k 2024-01-28T12:54:14.887 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 2.0 M 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 1.5 M 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 245 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 4.8 M 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 117 M 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-28T12:54:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-28T12:54:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-28T12:54:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-28T12:54:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-28T12:54:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-28T12:54:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-28T12:54:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-28T12:54:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-28T12:54:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:54:14.891 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================== 2024-01-28T12:54:14.892 INFO:teuthology.orchestra.run.smithi031.stdout:Install 36 Packages 2024-01-28T12:54:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:14.893 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 130 M 2024-01-28T12:54:14.893 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.4 G 2024-01-28T12:54:14.893 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:54:15.356 INFO:teuthology.orchestra.run.smithi031.stdout:(1/36): ceph-19.0.0-933.g00c0d54d.el9.x86_64.rp 29 kB/s | 6.5 kB 00:00 2024-01-28T12:54:15.798 INFO:teuthology.orchestra.run.smithi031.stdout:(2/36): ceph-mgr-19.0.0-933.g00c0d54d.el9.x86_6 2.2 MB/s | 1.5 MB 00:00 2024-01-28T12:54:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:(3/36): ceph-mds-19.0.0-933.g00c0d54d.el9.x86_6 2.7 MB/s | 2.0 MB 00:00 2024-01-28T12:54:15.965 INFO:teuthology.orchestra.run.smithi031.stdout:(4/36): ceph-mgr-modules-core-19.0.0-933.g00c0d 2.9 MB/s | 245 kB 00:00 2024-01-28T12:54:16.082 INFO:teuthology.orchestra.run.smithi031.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-01-28T12:54:16.124 INFO:teuthology.orchestra.run.smithi031.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-01-28T12:54:16.158 INFO:teuthology.orchestra.run.smithi031.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 944 kB/s | 32 kB 00:00 2024-01-28T12:54:16.199 INFO:teuthology.orchestra.run.smithi031.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-01-28T12:54:16.267 INFO:teuthology.orchestra.run.smithi031.stdout:(9/36): ceph-mon-19.0.0-933.g00c0d54d.el9.x86_6 5.3 MB/s | 4.8 MB 00:00 2024-01-28T12:54:16.292 INFO:teuthology.orchestra.run.smithi031.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 419 kB/s | 39 kB 00:00 2024-01-28T12:54:16.317 INFO:teuthology.orchestra.run.smithi031.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-01-28T12:54:16.376 INFO:teuthology.orchestra.run.smithi031.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2024-01-28T12:54:16.409 INFO:teuthology.orchestra.run.smithi031.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-01-28T12:54:16.435 INFO:teuthology.orchestra.run.smithi031.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.1 MB/s | 124 kB 00:00 2024-01-28T12:54:16.460 INFO:teuthology.orchestra.run.smithi031.stdout:(15/36): python3-requests-2.25.1-7.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-01-28T12:54:16.490 INFO:teuthology.orchestra.run.smithi031.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.4 MB/s | 187 kB 00:00 2024-01-28T12:54:16.573 INFO:teuthology.orchestra.run.smithi031.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 198 kB/s | 22 kB 00:00 2024-01-28T12:54:16.891 INFO:teuthology.orchestra.run.smithi031.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-01-28T12:54:16.924 INFO:teuthology.orchestra.run.smithi031.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 348 kB/s | 151 kB 00:00 2024-01-28T12:54:17.141 INFO:teuthology.orchestra.run.smithi031.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 686 kB/s | 172 kB 00:00 2024-01-28T12:54:17.241 INFO:teuthology.orchestra.run.smithi031.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-01-28T12:54:17.333 INFO:teuthology.orchestra.run.smithi031.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 193 kB/s | 18 kB 00:00 2024-01-28T12:54:17.433 INFO:teuthology.orchestra.run.smithi031.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-01-28T12:54:17.525 INFO:teuthology.orchestra.run.smithi031.stdout:(24/36): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-01-28T12:54:17.550 INFO:teuthology.orchestra.run.smithi031.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 572 kB/s | 358 kB 00:00 2024-01-28T12:54:17.617 INFO:teuthology.orchestra.run.smithi031.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 213 kB/s | 20 kB 00:00 2024-01-28T12:54:17.651 INFO:teuthology.orchestra.run.smithi031.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 460 kB/s | 46 kB 00:00 2024-01-28T12:54:17.710 INFO:teuthology.orchestra.run.smithi031.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 857 kB/s | 79 kB 00:00 2024-01-28T12:54:17.760 INFO:teuthology.orchestra.run.smithi031.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-01-28T12:54:17.802 INFO:teuthology.orchestra.run.smithi031.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-01-28T12:54:17.861 INFO:teuthology.orchestra.run.smithi031.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 894 kB/s | 90 kB 00:00 2024-01-28T12:54:17.886 INFO:teuthology.orchestra.run.smithi031.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 428 kB/s | 36 kB 00:00 2024-01-28T12:54:17.969 INFO:teuthology.orchestra.run.smithi031.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-01-28T12:54:18.078 INFO:teuthology.orchestra.run.smithi031.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.2 MB/s | 427 kB 00:00 2024-01-28T12:54:18.095 INFO:teuthology.orchestra.run.smithi031.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 159 kB/s | 20 kB 00:00 2024-01-28T12:54:22.787 INFO:teuthology.orchestra.run.smithi031.stdout:(36/36): ceph-osd-19.0.0-933.g00c0d54d.el9.x86_ 17 MB/s | 117 MB 00:06 2024-01-28T12:54:22.790 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:54:22.790 INFO:teuthology.orchestra.run.smithi031.stdout:Total 16 MB/s | 130 MB 00:07 2024-01-28T12:54:22.805 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:54:22.851 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:54:22.851 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:54:23.897 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:54:23.897 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:54:25.829 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:54:25.889 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-01-28T12:54:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-01-28T12:54:25.958 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-01-28T12:54:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-01-28T12:54:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-01-28T12:54:26.106 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-01-28T12:54:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-01-28T12:54:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-01-28T12:54:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-01-28T12:54:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-01-28T12:54:26.469 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-01-28T12:54:26.506 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 12/36 2024-01-28T12:54:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-01-28T12:54:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-01-28T12:54:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-01-28T12:54:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-01-28T12:54:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-01-28T12:54:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-01-28T12:54:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-01-28T12:54:27.034 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-01-28T12:54:27.074 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-01-28T12:54:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-01-28T12:54:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-01-28T12:54:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-01-28T12:54:27.225 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-01-28T12:54:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-01-28T12:54:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-01-28T12:54:27.447 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-01-28T12:54:27.616 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-01-28T12:54:27.743 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-01-28T12:54:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-933.g00c0d54d.el9.n 31/36 2024-01-28T12:54:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 32/36 2024-01-28T12:54:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 32/36 2024-01-28T12:54:27.977 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:27.977 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-28T12:54:27.977 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-28T12:54:27.977 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-28T12:54:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:28.897 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 33/36 2024-01-28T12:54:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 33/36 2024-01-28T12:54:28.923 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:28.923 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-28T12:54:28.923 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-28T12:54:28.923 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-28T12:54:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:29.406 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 34/36 2024-01-28T12:54:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 34/36 2024-01-28T12:54:29.432 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:29.432 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-28T12:54:29.432 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-28T12:54:29.432 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-28T12:54:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:29.640 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 35/36 2024-01-28T12:54:29.663 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 35/36 2024-01-28T12:54:29.663 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:29.664 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-28T12:54:29.664 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-28T12:54:29.664 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-28T12:54:29.664 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:29.730 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 36/36 2024-01-28T12:54:39.352 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 36/36 2024-01-28T12:54:39.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 1/36 2024-01-28T12:54:39.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 2/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 3/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 4/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 5/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-933.g00c0d54d.el9.n 6/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 16/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-01-28T12:54:39.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-01-28T12:54:39.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-01-28T12:54:39.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-01-28T12:54:40.298 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-28T12:54:40.299 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-28T12:54:40.300 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-28T12:54:40.301 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-28T12:54:40.301 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-28T12:54:40.301 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-28T12:54:40.301 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:40.301 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:54:40.554 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-base 2024-01-28T12:54:41.136 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:44 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:54:41.238 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:54:41.288 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:54:41.290 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:54:41.290 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:54:41.358 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install cephadm 2024-01-28T12:54:41.929 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:44 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 743 k 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-01-28T12:54:42.081 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:42.082 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 743 k 2024-01-28T12:54:42.082 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 748 k 2024-01-28T12:54:42.082 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:54:42.550 INFO:teuthology.orchestra.run.smithi137.stdout:cephadm-19.0.0-933.g00c0d54d.el9.noarch.rpm 1.6 MB/s | 743 kB 00:00 2024-01-28T12:54:42.551 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:54:42.551 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.5 MB/s | 743 kB 00:00 2024-01-28T12:54:42.551 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:54:42.556 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:54:42.556 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:54:42.561 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:54:42.561 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:54:42.631 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:54:43.094 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:54:43.179 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:54:43.572 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:54:43.901 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:54:43.901 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:43.901 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:54:43.901 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:54:43.901 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:43.901 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:54:44.146 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-immutable-object-cache 2024-01-28T12:54:44.741 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:47 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:54:44.892 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 147 k 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:54:44.893 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:44.894 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-01-28T12:54:44.894 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:44.894 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 147 k 2024-01-28T12:54:44.894 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 481 k 2024-01-28T12:54:44.894 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:54:45.114 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-immutable-object-cache-19.0.0-933.g00c0d54 669 kB/s | 147 kB 00:00 2024-01-28T12:54:45.114 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:54:45.114 INFO:teuthology.orchestra.run.smithi137.stdout:Total 665 kB/s | 147 kB 00:00 2024-01-28T12:54:45.114 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:54:45.122 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:54:45.122 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:54:45.175 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:54:45.175 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:54:45.311 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:54:45.373 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.e 1/1 2024-01-28T12:54:45.398 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.e 1/1 2024-01-28T12:54:45.398 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:45.398 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-28T12:54:45.398 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:46.471 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.e 1/1 2024-01-28T12:54:46.472 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:46.472 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:54:46.472 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:46.472 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:46.472 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:54:46.730 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr 2024-01-28T12:54:47.316 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:50 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:54:47.417 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:54:47.468 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:54:47.469 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:54:47.469 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:54:47.535 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-dashboard 2024-01-28T12:54:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 33/36 2024-01-28T12:54:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 33/36 2024-01-28T12:54:47.568 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:47.568 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-28T12:54:47.568 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-28T12:54:47.568 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-28T12:54:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:48.051 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 34/36 2024-01-28T12:54:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 34/36 2024-01-28T12:54:48.082 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:48.082 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-28T12:54:48.083 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-28T12:54:48.083 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-28T12:54:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:48.119 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:51 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:54:48.274 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 1.7 M 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 28 k 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 15 k 2024-01-28T12:54:48.275 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-28T12:54:48.276 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-28T12:54:48.276 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:48.276 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:54:48.276 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:54:48.276 INFO:teuthology.orchestra.run.smithi137.stdout:Install 5 Packages 2024-01-28T12:54:48.276 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:48.276 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 2.0 M 2024-01-28T12:54:48.277 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 8.5 M 2024-01-28T12:54:48.277 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:54:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 35/36 2024-01-28T12:54:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 35/36 2024-01-28T12:54:48.435 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:54:48.435 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-28T12:54:48.436 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-28T12:54:48.436 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-28T12:54:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 36/36 2024-01-28T12:54:48.568 INFO:teuthology.orchestra.run.smithi137.stdout:(1/5): ceph-grafana-dashboards-19.0.0-933.g00c0 118 kB/s | 28 kB 00:00 2024-01-28T12:54:48.593 INFO:teuthology.orchestra.run.smithi137.stdout:(2/5): ceph-prometheus-alerts-19.0.0-933.g00c0d 57 kB/s | 15 kB 00:00 2024-01-28T12:54:48.852 INFO:teuthology.orchestra.run.smithi137.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 108 kB/s | 31 kB 00:00 2024-01-28T12:54:49.002 INFO:teuthology.orchestra.run.smithi137.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 461 kB/s | 188 kB 00:00 2024-01-28T12:54:49.160 INFO:teuthology.orchestra.run.smithi137.stdout:(5/5): ceph-mgr-dashboard-19.0.0-933.g00c0d54d. 2.1 MB/s | 1.7 MB 00:00 2024-01-28T12:54:49.164 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:54:49.164 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2024-01-28T12:54:49.164 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:54:49.171 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:54:49.171 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:54:49.237 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:54:49.237 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:54:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:54:49.549 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-01-28T12:54:49.581 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-01-28T12:54:49.611 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-933.g00c0d54d.el9.no 3/5 2024-01-28T12:54:49.851 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-933.g00c0d54d.el9.n 4/5 2024-01-28T12:54:49.880 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 5/5 2024-01-28T12:54:50.437 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 5/5 2024-01-28T12:54:50.438 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-933.g00c0d54d.el9.n 1/5 2024-01-28T12:54:50.438 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 2/5 2024-01-28T12:54:50.438 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-933.g00c0d54d.el9.no 3/5 2024-01-28T12:54:50.438 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-01-28T12:54:50.726 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:50.727 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:54:50.956 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-diskprediction-local 2024-01-28T12:54:51.533 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:54 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:54:51.687 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:54:51.688 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================== 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================== 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 7.4 M 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-28T12:54:51.689 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-28T12:54:51.690 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-28T12:54:51.690 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-28T12:54:51.690 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:51.690 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:54:51.690 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================== 2024-01-28T12:54:51.690 INFO:teuthology.orchestra.run.smithi137.stdout:Install 12 Packages 2024-01-28T12:54:51.690 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:54:51.691 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 37 M 2024-01-28T12:54:51.691 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 209 M 2024-01-28T12:54:51.691 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:54:51.949 INFO:teuthology.orchestra.run.smithi137.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 32 kB 00:00 2024-01-28T12:54:51.974 INFO:teuthology.orchestra.run.smithi137.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 727 kB/s | 17 kB 00:00 2024-01-28T12:54:52.017 INFO:teuthology.orchestra.run.smithi137.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 813 kB/s | 34 kB 00:00 2024-01-28T12:54:52.067 INFO:teuthology.orchestra.run.smithi137.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-01-28T12:54:52.242 INFO:teuthology.orchestra.run.smithi137.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2024-01-28T12:54:52.318 INFO:teuthology.orchestra.run.smithi137.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-01-28T12:54:52.343 INFO:teuthology.orchestra.run.smithi137.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-01-28T12:54:52.543 INFO:teuthology.orchestra.run.smithi137.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 802 kB 00:00 2024-01-28T12:54:52.710 INFO:teuthology.orchestra.run.smithi137.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-01-28T12:54:52.744 INFO:teuthology.orchestra.run.smithi137.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 981 kB/s | 194 kB 00:00 2024-01-28T12:54:53.169 INFO:teuthology.orchestra.run.smithi137.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-9 5.5 MB/s | 7.4 MB 00:01 2024-01-28T12:54:53.586 INFO:teuthology.orchestra.run.smithi137.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 280 kB/s | 245 kB 00:00 2024-01-28T12:54:53.590 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:54:53.590 INFO:teuthology.orchestra.run.smithi137.stdout:Total 20 MB/s | 37 MB 00:01 2024-01-28T12:54:53.604 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:54:53.632 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:54:53.632 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:54:54.021 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:54:54.021 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:54:54.759 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:54:54.844 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-01-28T12:54:54.882 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-01-28T12:54:54.979 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-01-28T12:54:55.018 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-01-28T12:54:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-01-28T12:54:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-01-28T12:54:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-01-28T12:54:55.992 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-01-28T12:54:56.106 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-01-28T12:54:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-01-28T12:54:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 36/36 2024-01-28T12:54:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 1/36 2024-01-28T12:54:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 2/36 2024-01-28T12:54:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 3/36 2024-01-28T12:54:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 4/36 2024-01-28T12:54:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 5/36 2024-01-28T12:54:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-933.g00c0d54d.el9.n 6/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 16/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-01-28T12:54:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-01-28T12:54:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-01-28T12:54:58.382 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-01-28T12:54:58.408 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d5 12/12 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-28T12:54:59.006 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-28T12:54:59.007 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-28T12:54:59.008 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-28T12:54:59.009 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-28T12:54:59.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:54:59.009 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:54:59.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-01-28T12:54:59.876 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:00 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:54:59.981 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:00.035 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:00.036 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:00.036 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:00.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-01-28T12:55:00.677 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:01 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:00.837 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 743 k 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 743 k 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 748 k 2024-01-28T12:55:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d5 12/12 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d5 1/12 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-01-28T12:55:00.955 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-01-28T12:55:00.956 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-01-28T12:55:00.956 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-01-28T12:55:00.956 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-01-28T12:55:00.956 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-01-28T12:55:01.274 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-19.0.0-933.g00c0d54d.el9.noarch.rpm 1.7 MB/s | 743 kB 00:00 2024-01-28T12:55:01.275 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:01.275 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.7 MB/s | 743 kB 00:00 2024-01-28T12:55:01.275 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:01.281 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:01.281 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:01.287 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:01.287 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:01.436 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-01-28T12:55:01.437 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:01.437 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:01.437 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:01.437 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-28T12:55:01.522 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-28T12:55:01.522 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-28T12:55:01.522 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-28T12:55:01.522 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-28T12:55:01.522 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-28T12:55:01.523 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-28T12:55:01.523 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-28T12:55:01.523 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-28T12:55:01.523 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-28T12:55:01.523 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-28T12:55:01.523 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:01.523 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:55:01.856 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-rook 2024-01-28T12:55:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:55:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:55:02.443 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:05 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/1 2024-01-28T12:55:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:02.540 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:02.540 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:02.597 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:02.598 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-01-28T12:55:02.598 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:55:02.598 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-01-28T12:55:02.598 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:02.598 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 52 k 2024-01-28T12:55:02.598 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:55:02.598 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:02.599 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-01-28T12:55:02.600 INFO:teuthology.orchestra.run.smithi137.stdout:Install 13 Packages 2024-01-28T12:55:02.600 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:02.600 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 2.0 M 2024-01-28T12:55:02.600 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 26 M 2024-01-28T12:55:02.600 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:02.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-01-28T12:55:02.833 INFO:teuthology.orchestra.run.smithi137.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 161 kB/s | 24 kB 00:00 2024-01-28T12:55:02.858 INFO:teuthology.orchestra.run.smithi137.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 103 kB/s | 18 kB 00:00 2024-01-28T12:55:02.909 INFO:teuthology.orchestra.run.smithi137.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-01-28T12:55:02.934 INFO:teuthology.orchestra.run.smithi137.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-01-28T12:55:02.959 INFO:teuthology.orchestra.run.smithi137.stdout:(5/13): ceph-mgr-rook-19.0.0-933.g00c0d54d.el9. 188 kB/s | 52 kB 00:00 2024-01-28T12:55:02.984 INFO:teuthology.orchestra.run.smithi137.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2024-01-28T12:55:03.010 INFO:teuthology.orchestra.run.smithi137.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 568 kB/s | 43 kB 00:00 2024-01-28T12:55:03.337 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:04 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:03.494 INFO:teuthology.orchestra.run.smithi137.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 63 kB/s | 32 kB 00:00 2024-01-28T12:55:03.495 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:03.495 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 147 k 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-01-28T12:55:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:03.497 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 147 k 2024-01-28T12:55:03.497 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 481 k 2024-01-28T12:55:03.497 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:03.519 INFO:teuthology.orchestra.run.smithi137.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 28 kB/s | 15 kB 00:00 2024-01-28T12:55:03.661 INFO:teuthology.orchestra.run.smithi137.stdout:(10/13): python3-google-auth-2.26.2-1.el9.noarc 311 kB/s | 202 kB 00:00 2024-01-28T12:55:03.713 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-19.0.0-933.g00c0d54 680 kB/s | 147 kB 00:00 2024-01-28T12:55:03.714 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:03.714 INFO:teuthology.orchestra.run.smithi031.stdout:Total 675 kB/s | 147 kB 00:00 2024-01-28T12:55:03.714 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:03.722 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:03.722 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:03.775 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:03.775 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:03.844 INFO:teuthology.orchestra.run.smithi137.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 489 kB/s | 90 kB 00:00 2024-01-28T12:55:03.869 INFO:teuthology.orchestra.run.smithi137.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-01-28T12:55:03.926 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:03.989 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.e 1/1 2024-01-28T12:55:04.013 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.e 1/1 2024-01-28T12:55:04.014 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:55:04.014 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-28T12:55:04.014 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:04.036 INFO:teuthology.orchestra.run.smithi137.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-01-28T12:55:04.038 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:04.039 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-01-28T12:55:04.039 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:04.054 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:04.054 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:04.159 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:04.160 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:04.706 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:04.867 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-01-28T12:55:04.920 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-01-28T12:55:04.975 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-01-28T12:55:05.006 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-01-28T12:55:05.047 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-01-28T12:55:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.e 1/1 2024-01-28T12:55:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:05.077 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:05.077 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:05.139 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-01-28T12:55:05.256 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 7/13 2024-01-28T12:55:05.306 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-01-28T12:55:05.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-01-28T12:55:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:06 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:05.999 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:06.017 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-01-28T12:55:06.053 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:06.054 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:06.054 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-01-28T12:55:06.095 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-01-28T12:55:06.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-01-28T12:55:06.144 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-01-28T12:55:06.165 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 13/13 2024-01-28T12:55:06.706 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:07 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:06.865 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 1.7 M 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 28 k 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 15 k 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:Install 5 Packages 2024-01-28T12:55:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.0 M 2024-01-28T12:55:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 8.5 M 2024-01-28T12:55:06.867 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:06.981 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 13/13 2024-01-28T12:55:06.981 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 1/13 2024-01-28T12:55:06.981 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-01-28T12:55:06.981 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-01-28T12:55:06.981 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 10/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-01-28T12:55:06.982 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-01-28T12:55:07.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-01-28T12:55:07.350 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:07.350 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:07.350 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:07.351 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:07.609 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-cephadm 2024-01-28T12:55:07.758 INFO:teuthology.orchestra.run.smithi031.stdout:(1/5): ceph-grafana-dashboards-19.0.0-933.g00c0 57 kB/s | 28 kB 00:00 2024-01-28T12:55:07.792 INFO:teuthology.orchestra.run.smithi031.stdout:(2/5): ceph-prometheus-alerts-19.0.0-933.g00c0d 29 kB/s | 15 kB 00:00 2024-01-28T12:55:07.917 INFO:teuthology.orchestra.run.smithi031.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 194 kB/s | 31 kB 00:00 2024-01-28T12:55:08.008 INFO:teuthology.orchestra.run.smithi031.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 870 kB/s | 188 kB 00:00 2024-01-28T12:55:08.193 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:11 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:08.347 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:08.347 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 133 k 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-28T12:55:08.348 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:08.349 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:08.349 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-01-28T12:55:08.349 INFO:teuthology.orchestra.run.smithi137.stdout:Install 6 Packages 2024-01-28T12:55:08.349 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:08.349 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 6.8 M 2024-01-28T12:55:08.349 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 33 M 2024-01-28T12:55:08.349 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:08.484 INFO:teuthology.orchestra.run.smithi031.stdout:(5/5): ceph-mgr-dashboard-19.0.0-933.g00c0d54d. 1.4 MB/s | 1.7 MB 00:01 2024-01-28T12:55:08.486 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:08.486 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.2 MB/s | 2.0 MB 00:01 2024-01-28T12:55:08.486 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:08.495 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:08.495 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:08.565 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:08.565 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:08.713 INFO:teuthology.orchestra.run.smithi137.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.3 MB/s | 228 kB 00:00 2024-01-28T12:55:08.824 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-01-28T12:55:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-01-28T12:55:09.002 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-933.g00c0d54d.el9.no 3/5 2024-01-28T12:55:09.005 INFO:teuthology.orchestra.run.smithi137.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 5.8 MB 00:00 2024-01-28T12:55:09.138 INFO:teuthology.orchestra.run.smithi137.stdout:(3/6): ceph-mgr-cephadm-19.0.0-933.g00c0d54d.el 221 kB/s | 133 kB 00:00 2024-01-28T12:55:09.250 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-933.g00c0d54d.el9.n 4/5 2024-01-28T12:55:09.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 5/5 2024-01-28T12:55:09.455 INFO:teuthology.orchestra.run.smithi137.stdout:(4/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 739 kB/s | 548 kB 00:00 2024-01-28T12:55:09.480 INFO:teuthology.orchestra.run.smithi137.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 121 kB/s | 58 kB 00:00 2024-01-28T12:55:09.598 INFO:teuthology.orchestra.run.smithi137.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 111 kB/s | 51 kB 00:00 2024-01-28T12:55:09.602 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:09.602 INFO:teuthology.orchestra.run.smithi137.stdout:Total 5.4 MB/s | 6.8 MB 00:01 2024-01-28T12:55:09.602 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:09.611 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:09.611 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:09.700 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:09.700 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 5/5 2024-01-28T12:55:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-933.g00c0d54d.el9.n 1/5 2024-01-28T12:55:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 2/5 2024-01-28T12:55:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-933.g00c0d54d.el9.no 3/5 2024-01-28T12:55:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-01-28T12:55:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-01-28T12:55:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-28T12:55:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-28T12:55:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-01-28T12:55:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-01-28T12:55:10.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-01-28T12:55:10.739 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-01-28T12:55:10.827 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-01-28T12:55:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2024-01-28T12:55:10.920 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 6/6 2024-01-28T12:55:10.971 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:11 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:11.131 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================== 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================== 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 7.4 M 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-28T12:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================== 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-01-28T12:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 37 M 2024-01-28T12:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 209 M 2024-01-28T12:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:11.355 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 210 kB/s | 32 kB 00:00 2024-01-28T12:55:11.396 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 429 kB/s | 17 kB 00:00 2024-01-28T12:55:11.447 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 709 kB/s | 34 kB 00:00 2024-01-28T12:55:11.497 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-01-28T12:55:11.714 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2024-01-28T12:55:11.781 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-01-28T12:55:11.799 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 6/6 2024-01-28T12:55:11.799 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/6 2024-01-28T12:55:11.799 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-01-28T12:55:11.799 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2024-01-28T12:55:11.800 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-01-28T12:55:11.800 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-01-28T12:55:11.806 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-01-28T12:55:12.048 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 802 kB 00:00 2024-01-28T12:55:12.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-01-28T12:55:12.125 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:12.125 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:12.125 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-28T12:55:12.126 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-28T12:55:12.126 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-28T12:55:12.126 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-28T12:55:12.126 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-28T12:55:12.126 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:12.126 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:12.199 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-01-28T12:55:12.224 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 194 kB 00:00 2024-01-28T12:55:12.316 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-9 6.7 MB/s | 7.4 MB 00:01 2024-01-28T12:55:12.411 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-fuse 2024-01-28T12:55:12.474 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 893 kB/s | 245 kB 00:00 2024-01-28T12:55:12.477 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:12.478 INFO:teuthology.orchestra.run.smithi031.stdout:Total 28 MB/s | 37 MB 00:01 2024-01-28T12:55:12.491 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:12.515 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:12.516 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:12.914 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:12.914 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:12.992 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:15 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:13.145 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:13.145 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:13.145 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:55:13.145 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 858 k 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout:Install 2 Packages 2024-01-28T12:55:13.146 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:13.147 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 937 k 2024-01-28T12:55:13.147 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 3.1 M 2024-01-28T12:55:13.147 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:13.266 INFO:teuthology.orchestra.run.smithi137.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 662 kB/s | 79 kB 00:00 2024-01-28T12:55:13.499 INFO:teuthology.orchestra.run.smithi137.stdout:(2/2): ceph-fuse-19.0.0-933.g00c0d54d.el9.x86_6 2.4 MB/s | 858 kB 00:00 2024-01-28T12:55:13.501 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:13.501 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.6 MB/s | 937 kB 00:00 2024-01-28T12:55:13.501 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:13.521 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:13.521 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:13.596 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:13.596 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-01-28T12:55:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-01-28T12:55:13.782 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-01-28T12:55:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-01-28T12:55:13.918 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-01-28T12:55:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 2/2 2024-01-28T12:55:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-01-28T12:55:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-01-28T12:55:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-01-28T12:55:14.715 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 2/2 2024-01-28T12:55:14.715 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/2 2024-01-28T12:55:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-01-28T12:55:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-01-28T12:55:15.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-01-28T12:55:15.057 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:15.057 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:15.057 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-01-28T12:55:15.057 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:15.057 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:15.323 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-volume 2024-01-28T12:55:15.917 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:18 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:16.070 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 263 k 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout:Install 2 Packages 2024-01-28T12:55:16.071 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:16.072 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 332 k 2024-01-28T12:55:16.072 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 1.5 M 2024-01-28T12:55:16.072 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:16.187 INFO:teuthology.orchestra.run.smithi137.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 597 kB/s | 69 kB 00:00 2024-01-28T12:55:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-01-28T12:55:16.362 INFO:teuthology.orchestra.run.smithi137.stdout:(2/2): ceph-volume-19.0.0-933.g00c0d54d.el9.noa 907 kB/s | 263 kB 00:00 2024-01-28T12:55:16.363 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:16.364 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.1 MB/s | 332 kB 00:00 2024-01-28T12:55:16.364 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:16.372 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:16.372 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:16.413 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:16.413 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:16.558 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-01-28T12:55:16.781 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 2/2 2024-01-28T12:55:16.797 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 2/2 2024-01-28T12:55:16.836 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:55:16.836 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-28T12:55:16.836 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:17.401 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-01-28T12:55:17.429 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d5 12/12 2024-01-28T12:55:17.643 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 1/2 2024-01-28T12:55:17.954 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-01-28T12:55:17.954 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:17.955 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:17.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:17.955 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-01-28T12:55:17.955 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:17.955 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:18.305 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados-devel 2024-01-28T12:55:18.903 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:21 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:19.056 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 127 k 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:19.057 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-01-28T12:55:19.058 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:19.058 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 127 k 2024-01-28T12:55:19.058 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 457 k 2024-01-28T12:55:19.058 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:19.260 INFO:teuthology.orchestra.run.smithi137.stdout:librados-devel-19.0.0-933.g00c0d54d.el9.x86_64. 629 kB/s | 127 kB 00:00 2024-01-28T12:55:19.260 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:19.260 INFO:teuthology.orchestra.run.smithi137.stdout:Total 625 kB/s | 127 kB 00:00 2024-01-28T12:55:19.261 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:19.266 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:19.266 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:19.325 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:19.326 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:19.473 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:19.582 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:20.057 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:20.061 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d5 12/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d5 1/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-01-28T12:55:20.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-01-28T12:55:20.393 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:20.393 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:20.393 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:20.393 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:20.393 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:20.393 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:20.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-01-28T12:55:20.534 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:20.534 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:20.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:20.534 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-28T12:55:20.534 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-28T12:55:20.534 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:20.535 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:20.663 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs2 2024-01-28T12:55:20.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-01-28T12:55:21.256 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:24 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:21.357 INFO:teuthology.orchestra.run.smithi137.stdout:Package libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:21.411 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:21.412 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:55:21.412 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:21.484 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs-devel 2024-01-28T12:55:21.489 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:22 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:21.655 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 52 k 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-28T12:55:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-01-28T12:55:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:21.658 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.0 M 2024-01-28T12:55:21.658 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-01-28T12:55:21.658 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:21.953 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 140 kB/s | 18 kB 00:00 2024-01-28T12:55:21.978 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 158 kB/s | 24 kB 00:00 2024-01-28T12:55:22.029 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-01-28T12:55:22.062 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-01-28T12:55:22.068 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:25 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:22.088 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): ceph-mgr-rook-19.0.0-933.g00c0d54d.el9. 197 kB/s | 52 kB 00:00 2024-01-28T12:55:22.113 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-01-28T12:55:22.138 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-01-28T12:55:22.205 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 352 kB/s | 32 kB 00:00 2024-01-28T12:55:22.222 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 31 k 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:22.223 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:22.224 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-01-28T12:55:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:22.224 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 31 k 2024-01-28T12:55:22.224 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 140 k 2024-01-28T12:55:22.224 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:22.230 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 128 kB/s | 15 kB 00:00 2024-01-28T12:55:22.280 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-google-auth-2.26.2-1.el9.noarc 1.4 MB/s | 202 kB 00:00 2024-01-28T12:55:22.306 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-01-28T12:55:22.339 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-01-28T12:55:22.364 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-01-28T12:55:22.366 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:22.366 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.9 MB/s | 2.0 MB 00:00 2024-01-28T12:55:22.367 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:22.380 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:22.380 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:22.396 INFO:teuthology.orchestra.run.smithi137.stdout:libcephfs-devel-19.0.0-933.g00c0d54d.el9.x86_64 181 kB/s | 31 kB 00:00 2024-01-28T12:55:22.397 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:22.397 INFO:teuthology.orchestra.run.smithi137.stdout:Total 180 kB/s | 31 kB 00:00 2024-01-28T12:55:22.397 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:22.400 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:22.400 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:22.421 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:22.422 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:22.480 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:22.481 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-01-28T12:55:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-01-28T12:55:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:23.286 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:23.286 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-01-28T12:55:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-01-28T12:55:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-01-28T12:55:23.468 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados2 2024-01-28T12:55:23.497 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-01-28T12:55:23.606 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 7/13 2024-01-28T12:55:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-01-28T12:55:24.046 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:27 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:24.148 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:24.199 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:24.200 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:55:24.200 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:24.263 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librbd1 2024-01-28T12:55:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-01-28T12:55:24.473 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-01-28T12:55:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-01-28T12:55:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-01-28T12:55:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 13/13 2024-01-28T12:55:24.844 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:27 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:24.946 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:24.998 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:24.999 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:55:24.999 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:25.066 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rados 2024-01-28T12:55:25.371 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 13/13 2024-01-28T12:55:25.371 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 1/13 2024-01-28T12:55:25.371 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-01-28T12:55:25.371 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 10/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-01-28T12:55:25.372 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-01-28T12:55:25.640 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:28 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-01-28T12:55:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:25.740 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-28T12:55:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-28T12:55:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:25.742 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:25.742 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:25.792 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:25.793 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:55:25.793 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:25.859 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rgw 2024-01-28T12:55:26.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-01-28T12:55:26.432 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:29 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:26.533 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:26.584 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:26.584 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:55:26.585 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:26.612 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:27 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:26.646 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-cephfs 2024-01-28T12:55:26.772 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:26.773 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-01-28T12:55:26.773 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:55:26.773 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-01-28T12:55:26.773 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:26.773 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 133 k 2024-01-28T12:55:26.773 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:55:26.773 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-01-28T12:55:26.774 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 6.8 M 2024-01-28T12:55:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-01-28T12:55:26.775 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:27.082 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.4 MB/s | 228 kB 00:00 2024-01-28T12:55:27.166 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): ceph-mgr-cephadm-19.0.0-933.g00c0d54d.el 544 kB/s | 133 kB 00:00 2024-01-28T12:55:27.229 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:30 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:27.275 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-01-28T12:55:27.331 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:27.382 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:27.383 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:55:27.383 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:27.448 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rbd 2024-01-28T12:55:27.558 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 159 kB/s | 58 kB 00:00 2024-01-28T12:55:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 175 kB/s | 51 kB 00:00 2024-01-28T12:55:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-01-28T12:55:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.0 MB/s | 6.8 MB 00:00 2024-01-28T12:55:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:27.723 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:27.724 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:28.024 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:31 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:28.125 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:28.176 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:28.177 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-01-28T12:55:28.177 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-01-28T12:55:28.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-fuse 2024-01-28T12:55:28.252 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-01-28T12:55:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-01-28T12:55:28.819 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:31 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:28.898 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-01-28T12:55:28.981 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:28.982 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-01-28T12:55:28.982 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:28.982 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 86 k 2024-01-28T12:55:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 86 k 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 256 k 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:28.983 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2024-01-28T12:55:29.014 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 6/6 2024-01-28T12:55:29.199 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-fuse-19.0.0-933.g00c0d54d.el9.x86_64.rpm 397 kB/s | 86 kB 00:00 2024-01-28T12:55:29.200 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:29.200 INFO:teuthology.orchestra.run.smithi137.stdout:Total 394 kB/s | 86 kB 00:00 2024-01-28T12:55:29.200 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:29.208 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:29.208 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:29.265 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:29.265 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:29.409 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:29.828 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 6/6 2024-01-28T12:55:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 1/6 2024-01-28T12:55:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-01-28T12:55:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2024-01-28T12:55:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-01-28T12:55:29.829 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-01-28T12:55:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-01-28T12:55:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:30.156 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:30.157 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:30.337 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:30.337 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:30.337 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:30.337 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:30.337 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:30.337 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:30.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-01-28T12:55:30.566 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-mirror 2024-01-28T12:55:31.040 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:32 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:31.158 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:34 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:31.200 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:31.201 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:55:31.202 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:31.202 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 858 k 2024-01-28T12:55:31.202 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:55:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-28T12:55:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:31.202 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:31.203 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:31.203 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-01-28T12:55:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:31.203 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 937 k 2024-01-28T12:55:31.203 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 3.1 M 2024-01-28T12:55:31.203 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:31.317 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:31.317 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:31.317 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 3.0 M 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 3.0 M 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 14 M 2024-01-28T12:55:31.318 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:31.335 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 592 kB/s | 79 kB 00:00 2024-01-28T12:55:31.526 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-fuse-19.0.0-933.g00c0d54d.el9.x86_6 2.6 MB/s | 858 kB 00:00 2024-01-28T12:55:31.527 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:31.528 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.8 MB/s | 937 kB 00:00 2024-01-28T12:55:31.528 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:31.549 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:31.549 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:31.627 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:31.628 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:31.744 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-mirror-19.0.0-933.g00c0d54d.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2024-01-28T12:55:31.744 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:31.745 INFO:teuthology.orchestra.run.smithi137.stdout:Total 7.1 MB/s | 3.0 MB 00:00 2024-01-28T12:55:31.745 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:31.754 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:31.754 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:31.827 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:31.827 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-01-28T12:55:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 2/2 2024-01-28T12:55:32.258 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:32.312 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:32.336 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:32.337 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:55:32.337 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-28T12:55:32.337 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-28T12:55:32.337 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-28T12:55:32.337 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 2/2 2024-01-28T12:55:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/2 2024-01-28T12:55:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-01-28T12:55:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:33.058 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-01-28T12:55:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:33.059 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:33.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-volume 2024-01-28T12:55:33.495 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:33.496 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:33.496 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:33.496 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:33.496 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:33.496 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:33.722 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-nbd 2024-01-28T12:55:33.896 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:34 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:34.056 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.0.0-933.g00c0d54d.el9 ceph-noarch 263 k 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-01-28T12:55:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:34.058 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 332 k 2024-01-28T12:55:34.058 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.5 M 2024-01-28T12:55:34.058 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:34.181 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 562 kB/s | 69 kB 00:00 2024-01-28T12:55:34.316 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:37 ago on Sun 28 Jan 2024 12:52:57 PM UTC. 2024-01-28T12:55:34.423 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-volume-19.0.0-933.g00c0d54d.el9.noa 723 kB/s | 263 kB 00:00 2024-01-28T12:55:34.424 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:34.424 INFO:teuthology.orchestra.run.smithi031.stdout:Total 906 kB/s | 332 kB 00:00 2024-01-28T12:55:34.424 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:34.433 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:34.433 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:34.469 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 169 k 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-01-28T12:55:34.470 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:34.471 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 169 k 2024-01-28T12:55:34.471 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 540 k 2024-01-28T12:55:34.471 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-01-28T12:55:34.478 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:34.479 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:34.689 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-nbd-19.0.0-933.g00c0d54d.el9.x86_64.rpm 775 kB/s | 169 kB 00:00 2024-01-28T12:55:34.690 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:34.690 INFO:teuthology.orchestra.run.smithi137.stdout:Total 770 kB/s | 169 kB 00:00 2024-01-28T12:55:34.690 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-01-28T12:55:34.697 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-01-28T12:55:34.697 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-01-28T12:55:34.753 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-01-28T12:55:34.753 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-01-28T12:55:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-01-28T12:55:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 2/2 2024-01-28T12:55:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 2/2 2024-01-28T12:55:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:55:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-28T12:55:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:34.919 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-01-28T12:55:35.038 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:35.493 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:35.611 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 1/2 2024-01-28T12:55:35.793 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:35.793 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:35.793 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-01-28T12:55:35.793 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:35.793 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:35.793 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-01-28T12:55:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-01-28T12:55:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:35.944 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.0.0-933.g00c0d54d.el9.noarch 2024-01-28T12:55:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-01-28T12:55:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:35.945 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:36.070 DEBUG:teuthology.parallel:result is None 2024-01-28T12:55:36.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-01-28T12:55:36.901 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:37 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:37.062 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:37.062 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 127 k 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 457 k 2024-01-28T12:55:37.063 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:37.302 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-19.0.0-933.g00c0d54d.el9.x86_64. 531 kB/s | 127 kB 00:00 2024-01-28T12:55:37.303 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:37.303 INFO:teuthology.orchestra.run.smithi031.stdout:Total 528 kB/s | 127 kB 00:00 2024-01-28T12:55:37.303 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:37.309 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:37.309 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:38.403 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:38.403 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:38.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-01-28T12:55:39.263 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:40 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:39.368 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:39.422 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:39.423 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:39.424 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:39.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-01-28T12:55:40.082 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:41 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:40.242 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:40.242 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:40.242 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 31 k 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 31 k 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 140 k 2024-01-28T12:55:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-19.0.0-933.g00c0d54d.el9.x86_64 130 kB/s | 31 kB 00:00 2024-01-28T12:55:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:Total 130 kB/s | 31 kB 00:00 2024-01-28T12:55:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:40.487 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:40.487 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:40.515 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:40.515 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:41.437 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:41.437 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:41.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-01-28T12:55:42.211 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:43 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:42.315 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:42.370 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:42.371 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:42.371 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:42.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-01-28T12:55:43.024 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:44 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:43.129 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:43.183 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:43.184 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:43.184 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:43.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-01-28T12:55:43.843 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:44 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:44.001 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:44.002 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:44.002 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:44.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-01-28T12:55:44.662 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:45 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:44.767 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:44.821 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:44.822 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:44.823 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:44.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-01-28T12:55:45.477 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:46 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:45.581 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:45.636 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:45.637 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:45.637 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:45.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-01-28T12:55:46.296 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:47 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:46.400 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:19.0.0-933.g00c0d54d.el9.x86_64 is already installed. 2024-01-28T12:55:46.455 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:46.456 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-01-28T12:55:46.456 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:46.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-01-28T12:55:47.115 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:48 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:47.275 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 86 k 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-01-28T12:55:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:47.277 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 86 k 2024-01-28T12:55:47.277 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 256 k 2024-01-28T12:55:47.277 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:47.610 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-19.0.0-933.g00c0d54d.el9.x86_64.rpm 258 kB/s | 86 kB 00:00 2024-01-28T12:55:47.610 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:47.611 INFO:teuthology.orchestra.run.smithi031.stdout:Total 257 kB/s | 86 kB 00:00 2024-01-28T12:55:47.611 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:47.618 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:47.618 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:47.675 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:47.676 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:47.935 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:48.625 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:48.625 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:48.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-01-28T12:55:49.450 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:50 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:49.612 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 3.0 M 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:49.613 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:49.614 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-01-28T12:55:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:49.614 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.0 M 2024-01-28T12:55:49.614 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 14 M 2024-01-28T12:55:49.614 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:50.154 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-19.0.0-933.g00c0d54d.el9.x86_64.rpm 5.6 MB/s | 3.0 MB 00:00 2024-01-28T12:55:50.156 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:50.156 INFO:teuthology.orchestra.run.smithi031.stdout:Total 5.6 MB/s | 3.0 MB 00:00 2024-01-28T12:55:50.156 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:50.165 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:50.165 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:50.240 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:50.240 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:50.747 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:50.771 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-28T12:55:50.771 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-28T12:55:50.771 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-28T12:55:50.771 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-28T12:55:50.771 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:51.955 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:51.956 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:52.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-01-28T12:55:52.787 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:53 ago on Sun 28 Jan 2024 12:52:59 PM UTC. 2024-01-28T12:55:52.947 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-01-28T12:55:52.947 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.0.0-933.g00c0d54d.el9 ceph 169 k 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 169 k 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 540 k 2024-01-28T12:55:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-01-28T12:55:53.682 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-19.0.0-933.g00c0d54d.el9.x86_64.rpm 230 kB/s | 169 kB 00:00 2024-01-28T12:55:53.683 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-01-28T12:55:53.683 INFO:teuthology.orchestra.run.smithi031.stdout:Total 230 kB/s | 169 kB 00:00 2024-01-28T12:55:53.683 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-01-28T12:55:53.692 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-01-28T12:55:53.692 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-01-28T12:55:53.753 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-01-28T12:55:53.753 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-01-28T12:55:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-01-28T12:55:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:54.500 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:54.851 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 1/1 2024-01-28T12:55:54.851 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:54.851 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-01-28T12:55:54.852 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.0.0-933.g00c0d54d.el9.x86_64 2024-01-28T12:55:54.852 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:54.852 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-01-28T12:55:55.176 DEBUG:teuthology.parallel:result is None 2024-01-28T12:55:55.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=00c0d54d57237e78360214c1fefe250a2610dcfe 2024-01-28T12:55:55.317 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-28T12:55:55.338 INFO:teuthology.orchestra.run.smithi031.stdout:19.0.0-933.g00c0d54d.el9 2024-01-28T12:55:55.338 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-933.g00c0d54d.el9 2024-01-28T12:55:55.339 INFO:teuthology.task.install:The correct ceph version 19.0.0-933.g00c0d54d is installed. 2024-01-28T12:55:55.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=00c0d54d57237e78360214c1fefe250a2610dcfe 2024-01-28T12:55:55.477 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-28T12:55:55.501 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-933.g00c0d54d.el9 2024-01-28T12:55:55.502 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-933.g00c0d54d.el9 2024-01-28T12:55:55.502 INFO:teuthology.task.install:The correct ceph version 19.0.0-933.g00c0d54d is installed. 2024-01-28T12:55:55.504 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-28T12:55:55.504 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:55.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-28T12:55:55.533 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:55:55.533 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-28T12:55:55.572 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-28T12:55:55.572 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:55.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-01-28T12:55:55.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-28T12:55:55.667 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:55:55.667 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-01-28T12:55:55.699 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-28T12:55:55.770 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-28T12:55:55.771 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:55.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-28T12:55:55.798 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-28T12:55:55.865 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:55:55.865 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-28T12:55:55.895 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-28T12:55:55.964 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-28T12:55:55.964 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:55.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-01-28T12:55:55.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-28T12:55:56.059 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:55:56.059 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-01-28T12:55:56.088 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-28T12:55:56.157 INFO:teuthology.run_tasks:Running task ceph... 2024-01-28T12:55:56.241 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-28T12:55:56.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-01-28T12:55:56.244 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /var/log/ceph 2024-01-28T12:55:56.272 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-28T12:55:56.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-28T12:55:56.310 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-28T12:55:56.340 INFO:tasks.ceph:Creating extra log directories... 2024-01-28T12:55:56.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-28T12:55:56.376 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-28T12:55:56.418 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-28T12:55:56.418 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson osd obc lru size': 10, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-28T12:55:56.418 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799', 'branch': 'wip-matanb-crimson-pg-map-logs', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7535799', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-pg-map-logs'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson osd obc lru size': 10, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '7eb9e33f53ab64f23bcf0a2a31509f3442110e4e'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 7326, 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '7eb9e33f53ab64f23bcf0a2a31509f3442110e4e', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+e/mN47WL+k2UShx4+QdYe2PoCv9kFTOWTk6NUp62n3KES5c+ZwdMP9U4M1mNve/sZ1c2dU0QAIgqEU58p7p8=', 'smithi137.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIydCY7OmjmfsVwUzUdkkPVrOiO0zWzLJGxQNS0ZorlF1m7YkqFZ0KHhg6Emxon2MVNwuKOiZXxfyftCEFTDDvA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson osd obc lru size': 10, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '00c0d54d57237e78360214c1fefe250a2610dcfe', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c856e0f9244dd29a50697123b480d63f16a4ed81', 'timestamp': '2024-01-28_12:38:29', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-28T12:55:56.419 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-28T12:55:56.443 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-28T12:55:56.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-01-28T12:55:56.498 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/run/ceph 2024-01-28T12:55:56.530 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:56.530 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-01-28T12:55:56.576 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-28T12:55:56.576 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-01-28T12:55:56.630 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-28 12:55:51.012976746 +0000 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-28 12:52:13.324319701 +0000 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-28 12:52:13.324319701 +0000 2024-01-28T12:55:56.631 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-28 12:52:13.324319701 +0000 2024-01-28T12:55:56.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-28T12:55:56.696 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-28T12:55:56.696 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-28T12:55:56.696 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000237082 s, 2.2 MB/s 2024-01-28T12:55:56.697 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-28T12:55:56.752 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-01-28T12:55:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-28T12:55:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:56.807 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-01-28T12:55:56.807 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:56.807 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-28 12:55:51.013976730 +0000 2024-01-28T12:55:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-28 12:52:13.688313768 +0000 2024-01-28T12:55:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-28 12:52:13.688313768 +0000 2024-01-28T12:55:56.808 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-28 12:52:13.688313768 +0000 2024-01-28T12:55:56.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-28T12:55:56.873 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-28T12:55:56.873 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-28T12:55:56.873 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000164566 s, 3.1 MB/s 2024-01-28T12:55:56.874 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-28T12:55:56.929 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-01-28T12:55:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-28T12:55:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:56.984 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-01-28T12:55:56.984 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:56.984 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:56.984 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-28 12:55:51.013976730 +0000 2024-01-28T12:55:56.984 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-28 12:52:14.027308242 +0000 2024-01-28T12:55:56.985 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-28 12:52:14.027308242 +0000 2024-01-28T12:55:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-28 12:52:14.027308242 +0000 2024-01-28T12:55:56.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-28T12:55:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-28T12:55:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-28T12:55:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000170207 s, 3.0 MB/s 2024-01-28T12:55:57.051 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-28T12:55:57.107 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-01-28 12:55:51.013976730 +0000 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-01-28 12:52:14.383302439 +0000 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-01-28 12:52:14.383302439 +0000 2024-01-28T12:55:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-01-28 12:52:14.383302439 +0000 2024-01-28T12:55:57.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-28T12:55:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-01-28T12:55:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-01-28T12:55:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000211926 s, 2.4 MB/s 2024-01-28T12:55:57.227 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-28T12:55:57.282 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-01-28T12:55:57.282 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:55:57.282 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-01-28T12:55:57.301 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-28T12:55:57.301 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-01-28T12:55:57.358 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-28 12:55:32.581736674 +0000 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-28 12:52:10.347105333 +0000 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-28 12:52:10.347105333 +0000 2024-01-28T12:55:57.359 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-28 12:52:10.347105333 +0000 2024-01-28T12:55:57.359 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-28T12:55:57.428 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-28T12:55:57.428 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-28T12:55:57.428 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000219027 s, 2.3 MB/s 2024-01-28T12:55:57.429 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-28T12:55:57.488 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-01-28T12:55:57.545 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-28T12:55:57.545 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:57.545 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-01-28T12:55:57.545 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:57.545 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:57.546 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-28 12:55:32.581736674 +0000 2024-01-28T12:55:57.546 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-28 12:52:10.706098907 +0000 2024-01-28T12:55:57.546 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-28 12:52:10.706098907 +0000 2024-01-28T12:55:57.546 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-28 12:52:10.706098907 +0000 2024-01-28T12:55:57.546 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-28T12:55:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-28T12:55:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-28T12:55:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.00017234 s, 3.0 MB/s 2024-01-28T12:55:57.615 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-28T12:55:57.672 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-28 12:55:32.581736674 +0000 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-28 12:52:11.040092929 +0000 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-28 12:52:11.040092929 +0000 2024-01-28T12:55:57.728 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-28 12:52:11.040092929 +0000 2024-01-28T12:55:57.729 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-28T12:55:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-28T12:55:57.795 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-28T12:55:57.795 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000226836 s, 2.3 MB/s 2024-01-28T12:55:57.796 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-28T12:55:57.852 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-01-28T12:55:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-28T12:55:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-28T12:55:57.907 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-01-28T12:55:57.907 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-28T12:55:57.907 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-01-28T12:55:57.907 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-01-28 12:55:32.582736656 +0000 2024-01-28T12:55:57.907 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-01-28 12:52:11.385086753 +0000 2024-01-28T12:55:57.908 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-01-28 12:52:11.385086753 +0000 2024-01-28T12:55:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-01-28 12:52:11.385086753 +0000 2024-01-28T12:55:57.908 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-28T12:55:57.973 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-01-28T12:55:57.973 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-01-28T12:55:57.973 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000184603 s, 2.8 MB/s 2024-01-28T12:55:57.974 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-28T12:55:58.030 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-01-28T12:55:58.030 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-01-28T12:55:58.030 INFO:tasks.ceph:Generating config... 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-28T12:55:58.032 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-28T12:55:58.033 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-01-28T12:55:58.033 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] debug monc = 20 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] crimson seastar smp = 3 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] osd class default list = * 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] osd class load list = * 2024-01-28T12:55:58.033 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-01-28T12:55:58.033 INFO:tasks.ceph:[client] debug ms = 1 2024-01-28T12:55:58.033 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-01-28T12:55:58.033 INFO:tasks.ceph:[global] ms client mode = crc 2024-01-28T12:55:58.034 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-01-28T12:55:58.034 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-01-28T12:55:58.034 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-01-28T12:55:58.034 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-01-28T12:55:58.034 INFO:tasks.ceph:[global] ms service mode = crc 2024-01-28T12:55:58.034 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-28T12:55:58.034 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-28T12:55:58.034 INFO:tasks.ceph:Setting up mon.a... 2024-01-28T12:55:58.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-28T12:55:58.099 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-01-28T12:55:58.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-28T12:55:58.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-28T12:55:58.181 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31')] 2024-01-28T12:55:58.182 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson osd obc lru size': 10, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}} 2024-01-28T12:55:58.182 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:58.183 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-28T12:55:58.236 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --print /home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:55:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:55:58.323+0000 7fbb6084ee80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:55:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:55:58.324+0000 7fbb6084ee80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:55:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:55:58.324+0000 7fbb6084ee80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:55:58.322 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:55:58.322 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:55:58.322 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-01-28T12:55:58.322 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-01-28T12:55:58.323 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:55:58.323 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-01-28T12:55:58.325388+0000 2024-01-28T12:55:58.323 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-01-28T12:55:58.325388+0000 2024-01-28T12:55:58.323 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 17 (quincy) 2024-01-28T12:55:58.323 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-01-28T12:55:58.323 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-01-28T12:55:58.323 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-01-28T12:55:58.324 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-28T12:55:58.338 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 68a362e5-2887-44cb-94c6-4a7b059316bf... 2024-01-28T12:55:58.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-28T12:55:58.381 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-28T12:55:58.428 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-01-28T12:55:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-01-28T12:55:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-01-28T12:55:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = 172.21.15.31 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms client mode = crc 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms cluster mode = crc 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms mon client mode = crc 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms mon cluster mode = crc 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms mon service mode = crc 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: ms service mode = crc 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:55:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-01-28T12:55:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-01-28T12:55:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: debug monc = 20 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: crimson osd obc lru size = 10 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: crimson seastar smp = 3 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-01-28T12:55:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: osd class default list = * 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: osd class load list = * 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: osd shutdown pgref assert = True 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-01-28T12:55:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-01-28T12:55:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: mon min osdmap epochs = 50 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: mon osdmap full prune interval = 2 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: mon osdmap full prune min = 15 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: mon osdmap full prune txsize = 2 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: paxos service trim min = 10 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on pool no app = False 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default crimson = True 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-01-28T12:55:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-01-28T12:55:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-01-28T12:55:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-28T12:55:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-28T12:55:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-01-28T12:55:58.440 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: chdir = "" 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: auth supported = cephx 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: filestore xattr use omap = true 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: mon clock drift allowed = 1.000 2024-01-28T12:55:58.441 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: osd crush chooseleaf type = 0 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: auth debug = true 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on old message = true 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on bug = true 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: mon pg warn max object skew = 0 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: # disable pg_autoscaler by default for new pools 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default size = 2 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.442 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow primary affinity = true 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow pg remap = true 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on legacy crush tunables = false 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on crush straw calc version zero = false 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on no sortbitwise = false 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on osd down out interval zero = false 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on too few osds = false 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon_allow_pool_size_one = true 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: osd default data pool replay window = 5 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.443 INFO:teuthology.orchestra.run.smithi137.stdout: mon allow pool delete = true 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: mon cluster log file level = debug 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: debug asserts on shutdown = true 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: mon health detail to clog = false 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: mon host = 172.21.15.31 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: ms client mode = crc 2024-01-28T12:55:58.444 INFO:teuthology.orchestra.run.smithi137.stdout: ms cluster mode = crc 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: ms mon client mode = crc 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: ms mon cluster mode = crc 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: ms mon service mode = crc 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: ms service mode = crc 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout:[osd] 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: osd journal size = 100 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub load threshold = 5.0 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub max interval = 600 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: osd mclock profile = high_recovery_ops 2024-01-28T12:55:58.445 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd recover clone overlap = true 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd recovery max chunk = 1048576 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug shutdown = true 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug op order = true 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify stray on activate = true 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug trim objects = true 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd open classes on start = true 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug pg log writeout = true 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd deep scrub update digest min age = 30 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.446 INFO:teuthology.orchestra.run.smithi137.stdout: osd map max advance = 10 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: journal zero on create = true 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: filestore ondisk finisher threads = 3 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: filestore apply finisher threads = 3 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: bdev debug aio = true 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug misdirected ops = true 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: debug monc = 20 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: crimson osd obc lru size = 10 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: crimson seastar smp = 3 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: debug osd = 20 2024-01-28T12:55:58.447 INFO:teuthology.orchestra.run.smithi137.stdout: osd blocked scrub grace period = 3600 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: osd class default list = * 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: osd class load list = * 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: osd shutdown pgref assert = True 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr] 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: debug mgr = 20 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-01-28T12:55:58.448 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mgr/telemetry/nag = false 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout:[mon] 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: debug paxos = 20 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon data avail warn = 5 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon mgr mkfs grace = 240 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd reporter subtree level = osd 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd prime pg temp = true 2024-01-28T12:55:58.449 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: auth mon ticket ttl = 660 # 11m 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: auth service ticket ttl = 240 # 4m 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: # don't complain about insecure global_id in the test suite 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: # 1m isn't quite enough 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon_down_mkfs_grace = 2m 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_filestore_osds = false 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon min osdmap epochs = 50 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon osdmap full prune interval = 2 2024-01-28T12:55:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: mon osdmap full prune min = 15 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon osdmap full prune txsize = 2 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: paxos service trim min = 10 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on pool no app = False 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default crimson = True 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout:[client] 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: rgw cache enabled = true 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable ops log = true 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable usage log = true 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-01-28T12:55:58.451 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.a] 2024-01-28T12:55:58.453 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-28T12:55:58.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-28T12:55:58.553 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-28T12:55:58.553 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:58.553 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-28T12:55:58.567 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:58.568 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-28T12:55:58.622 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-01-28T12:55:58.622 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:58.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-28T12:55:58.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-28T12:55:58.701 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:55:58.701 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:55:58.755 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi137.front.sepia.ceph.com 2024-01-28T12:55:58.755 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:55:58.755 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-28T12:55:58.755 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-28T12:55:58.796 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:55:58.796 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:55:58.851 INFO:tasks.ceph:Setting up mon nodes... 2024-01-28T12:55:58.851 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-28T12:55:58.851 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-28T12:55:58.971 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-28T12:55:58.974 INFO:tasks.ceph:Setting up mds nodes... 2024-01-28T12:55:58.974 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-28T12:55:58.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-28T12:55:59.023 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-28T12:55:59.039 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-28T12:55:59.087 INFO:teuthology.orchestra.run.smithi137.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-28T12:55:59.102 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-28T12:55:59.103 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-01-28T12:55:59.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-28T12:55:59.130 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-01-28T12:55:59.130 INFO:tasks.ceph:role: osd.0 2024-01-28T12:55:59.130 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-28T12:55:59.130 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-28T12:55:59.204 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-28T12:55:59.720 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-01-28T12:55:59.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-01-28T12:55:59.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-28T12:55:59.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-28T12:55:59.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-28T12:55:59.964 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-01-28T12:55:59.965 INFO:tasks.ceph:role: osd.1 2024-01-28T12:55:59.965 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-28T12:55:59.965 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-28T12:56:00.040 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-28T12:56:00.040 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-28T12:56:00.040 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-28T12:56:00.041 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-28T12:56:00.041 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-28T12:56:00.041 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-01-28T12:56:00.041 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-28T12:56:00.041 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-28T12:56:00.041 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-28T12:56:00.041 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-28T12:56:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-01-28T12:56:00.512 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-01-28T12:56:00.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-28T12:56:00.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-28T12:56:00.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:56:01.168 INFO:teuthology.orchestra.run.smithi031.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:01.168 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,172 seastar - Reactor backend: linux-aio 2024-01-28T12:56:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,191 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,191 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,191 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:01.189 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,191 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:==28156==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:==28156==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff3656f000; bottom 0x7fb094019000; size: 0x004ea2556000 (337730953216) 2024-01-28T12:56:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:False positive error reports may follow 2024-01-28T12:56:01.197 INFO:teuthology.orchestra.run.smithi031.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:01.223 INFO:teuthology.orchestra.run.smithi031.stderr:Reactor stalled for 32 ms on shard 0. Backtrace: 0x45d5d 0x2e8cd756 0x2e8ceb04 0x2e8d0036 0x2e8d03ba 0x2e8d0504 0x2e8d095a 0x54daf 0x241960e6 0x24196589 0x2419c135 0x23eff768 0x23f51523 0x25f3afa4 0x25f42ad8 0x25f45425 0x25f465ea 0x23a10523 0x23a6c164 0x23a9dc01 0x2e84010f 0x2e8a3340 0x2eb49287 0x2eb4b0d1 0x2e4fc60e 0x2e4fe517 0x1f9b7275 0x1f9bdd1d 0x1f796b05 0x3feaf 0x3ff5f 0x1f357ba4 2024-01-28T12:56:01.223 INFO:teuthology.orchestra.run.smithi031.stderr:kernel callstack: 2024-01-28T12:56:01.233 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,236 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2024-01-28T12:56:01.234 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,236 [shard 0] osd - get_early_config: set --smp 3 2024-01-28T12:56:01.749 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:01.749 INFO:teuthology.orchestra.run.smithi031.stderr:Suppressions used: 2024-01-28T12:56:01.750 INFO:teuthology.orchestra.run.smithi031.stderr: count bytes template 2024-01-28T12:56:01.750 INFO:teuthology.orchestra.run.smithi031.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:01.750 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:01.750 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-28T12:56:01.799 INFO:teuthology.orchestra.run.smithi031.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:01.800 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,803 seastar - Reactor backend: linux-aio 2024-01-28T12:56:01.818 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,821 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:01.818 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,821 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:01.818 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,821 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:01.825 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:01,828 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:01.828 INFO:teuthology.orchestra.run.smithi031.stderr:==28154==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:==28154==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff3656f000; bottom 0x7fb091e02000; size: 0x004ea476d000 (337766699008) 2024-01-28T12:56:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:False positive error reports may follow 2024-01-28T12:56:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:04.923 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:Suppressions used: 2024-01-28T12:56:04.924 INFO:teuthology.orchestra.run.smithi031.stderr: count bytes template 2024-01-28T12:56:04.924 INFO:teuthology.orchestra.run.smithi031.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:04.924 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-28T12:56:04.924 INFO:teuthology.orchestra.run.smithi031.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:56:04.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-28T12:56:04.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:56:05.523 INFO:teuthology.orchestra.run.smithi031.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:05.523 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:05,526 seastar - Reactor backend: linux-aio 2024-01-28T12:56:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:05,546 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:05,546 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:05,546 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:05,546 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:05.545 INFO:teuthology.orchestra.run.smithi031.stderr:==28964==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:05.551 INFO:teuthology.orchestra.run.smithi031.stderr:==28964==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe47955000; bottom 0x7fa278efe000; size: 0x005bcea57000 (394308972544) 2024-01-28T12:56:05.552 INFO:teuthology.orchestra.run.smithi031.stderr:False positive error reports may follow 2024-01-28T12:56:05.552 INFO:teuthology.orchestra.run.smithi031.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:05.578 INFO:teuthology.orchestra.run.smithi031.stderr:Reactor stalled for 32 ms on shard 0. Backtrace: 0x45d5d 0x2e8cd756 0x2e8ceb04 0x2e8d0036 0x2e8d03ba 0x2e8d0504 0x2e8d095a 0x54daf 0x9449b 0x94d46 0x2419b303 0x23eff768 0x23f51523 0x25f3afa4 0x25f42ad8 0x25f45425 0x25f465ea 0x23a10523 0x23a6c164 0x23a9dc01 0x2e84010f 0x2e8a3340 0x2eb49287 0x2eb4b0d1 0x2e4fc60e 0x2e4fe517 0x1f9b7275 0x1f9bdd1d 0x1f796b05 0x3feaf 0x3ff5f 0x1f357ba4 2024-01-28T12:56:05.578 INFO:teuthology.orchestra.run.smithi031.stderr:kernel callstack: 2024-01-28T12:56:05.588 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:05,591 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2024-01-28T12:56:05.588 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:05,591 [shard 0] osd - get_early_config: set --smp 3 2024-01-28T12:56:06.087 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:06.087 INFO:teuthology.orchestra.run.smithi031.stderr:Suppressions used: 2024-01-28T12:56:06.087 INFO:teuthology.orchestra.run.smithi031.stderr: count bytes template 2024-01-28T12:56:06.087 INFO:teuthology.orchestra.run.smithi031.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:06.087 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:06.087 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-28T12:56:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:06.144 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:06,147 seastar - Reactor backend: linux-aio 2024-01-28T12:56:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:06,165 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:06,165 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:06,165 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:INFO 2024-01-28 12:56:06,173 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:==28962==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:==28962==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe47955000; bottom 0x7fa276ce7000; size: 0x005bd0c6e000 (394344718336) 2024-01-28T12:56:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:False positive error reports may follow 2024-01-28T12:56:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:Suppressions used: 2024-01-28T12:56:09.283 INFO:teuthology.orchestra.run.smithi031.stderr: count bytes template 2024-01-28T12:56:09.283 INFO:teuthology.orchestra.run.smithi031.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:----------------------------------------------------- 2024-01-28T12:56:09.283 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-28T12:56:09.283 INFO:teuthology.orchestra.run.smithi031.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:56:09.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-28T12:56:09.352 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-28T12:56:09.384 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-01-28T12:56:09.384 INFO:tasks.ceph:role: osd.2 2024-01-28T12:56:09.385 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-28T12:56:09.385 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-28T12:56:09.463 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-28T12:56:09.464 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-28T12:56:09.975 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-01-28T12:56:09.977 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-01-28T12:56:09.977 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-01-28T12:56:10.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-28T12:56:10.174 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-28T12:56:10.245 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-01-28T12:56:10.246 INFO:tasks.ceph:role: osd.3 2024-01-28T12:56:10.246 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-28T12:56:10.246 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-28T12:56:10.323 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-28T12:56:10.324 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-28T12:56:10.773 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-01-28T12:56:10.775 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-01-28T12:56:10.775 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-01-28T12:56:10.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-28T12:56:10.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:56:11.447 INFO:teuthology.orchestra.run.smithi137.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:11.448 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:11,451 seastar - Reactor backend: linux-aio 2024-01-28T12:56:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:11,468 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:11,468 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:11,468 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:11,468 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:11.467 INFO:teuthology.orchestra.run.smithi137.stderr:==28064==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:11.474 INFO:teuthology.orchestra.run.smithi137.stderr:==28064==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc6ca6a000; bottom 0x7f004fff1000; size: 0x00fc1ca79000 (1082812502016) 2024-01-28T12:56:11.474 INFO:teuthology.orchestra.run.smithi137.stderr:False positive error reports may follow 2024-01-28T12:56:11.474 INFO:teuthology.orchestra.run.smithi137.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:11.510 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:11,513 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2024-01-28T12:56:11.510 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:11,513 [shard 0] osd - get_early_config: set --smp 3 2024-01-28T12:56:11.994 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:11.994 INFO:teuthology.orchestra.run.smithi137.stderr:Suppressions used: 2024-01-28T12:56:11.994 INFO:teuthology.orchestra.run.smithi137.stderr: count bytes template 2024-01-28T12:56:11.994 INFO:teuthology.orchestra.run.smithi137.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:11.994 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:11.994 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-01-28T12:56:12.043 INFO:teuthology.orchestra.run.smithi137.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:12.044 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:12,047 seastar - Reactor backend: linux-aio 2024-01-28T12:56:12.062 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:12,064 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:12.062 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:12,065 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:12.062 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:12,065 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:12,072 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:==28062==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:12.078 INFO:teuthology.orchestra.run.smithi137.stderr:==28062==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc6ca6a000; bottom 0x7f004ddda000; size: 0x00fc1ec90000 (1082848247808) 2024-01-28T12:56:12.078 INFO:teuthology.orchestra.run.smithi137.stderr:False positive error reports may follow 2024-01-28T12:56:12.078 INFO:teuthology.orchestra.run.smithi137.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:15.120 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:15.120 INFO:teuthology.orchestra.run.smithi137.stderr:Suppressions used: 2024-01-28T12:56:15.120 INFO:teuthology.orchestra.run.smithi137.stderr: count bytes template 2024-01-28T12:56:15.121 INFO:teuthology.orchestra.run.smithi137.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:15.121 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:15.121 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-01-28T12:56:15.121 INFO:teuthology.orchestra.run.smithi137.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:56:15.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-28T12:56:15.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:56:15.708 INFO:teuthology.orchestra.run.smithi137.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:15.709 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:15,712 seastar - Reactor backend: linux-aio 2024-01-28T12:56:15.725 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:15,728 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:15.725 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:15,728 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:15.725 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:15,728 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:15.725 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:15,728 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:15.727 INFO:teuthology.orchestra.run.smithi137.stderr:==28873==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:15.733 INFO:teuthology.orchestra.run.smithi137.stderr:==28873==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe988e1000; bottom 0x7f1f60427000; size: 0x00df384ba000 (958722187264) 2024-01-28T12:56:15.733 INFO:teuthology.orchestra.run.smithi137.stderr:False positive error reports may follow 2024-01-28T12:56:15.733 INFO:teuthology.orchestra.run.smithi137.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:15.768 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:15,771 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2024-01-28T12:56:15.768 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:15,771 [shard 0] osd - get_early_config: set --smp 3 2024-01-28T12:56:16.235 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:16.235 INFO:teuthology.orchestra.run.smithi137.stderr:Suppressions used: 2024-01-28T12:56:16.236 INFO:teuthology.orchestra.run.smithi137.stderr: count bytes template 2024-01-28T12:56:16.236 INFO:teuthology.orchestra.run.smithi137.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:16.236 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:16.236 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-01-28T12:56:16.287 INFO:teuthology.orchestra.run.smithi137.stderr:WARNING: debug mode. Not for benchmarking or production 2024-01-28T12:56:16.287 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:16,291 seastar - Reactor backend: linux-aio 2024-01-28T12:56:16.315 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:16,318 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2024-01-28T12:56:16.315 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:16,318 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2024-01-28T12:56:16.315 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:16,318 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2024-01-28T12:56:16.323 INFO:teuthology.orchestra.run.smithi137.stderr:INFO 2024-01-28 12:56:16,326 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2024-01-28T12:56:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:==28871==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-01-28T12:56:16.331 INFO:teuthology.orchestra.run.smithi137.stderr:==28871==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe988e1000; bottom 0x7f1f5e210000; size: 0x00df3a6d1000 (958757933056) 2024-01-28T12:56:16.331 INFO:teuthology.orchestra.run.smithi137.stderr:False positive error reports may follow 2024-01-28T12:56:16.331 INFO:teuthology.orchestra.run.smithi137.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-01-28T12:56:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:Suppressions used: 2024-01-28T12:56:19.407 INFO:teuthology.orchestra.run.smithi137.stderr: count bytes template 2024-01-28T12:56:19.407 INFO:teuthology.orchestra.run.smithi137.stderr: 1 8 MallocExtension::Register 2024-01-28T12:56:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:----------------------------------------------------- 2024-01-28T12:56:19.407 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-01-28T12:56:19.408 INFO:teuthology.orchestra.run.smithi137.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T12:56:19.449 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-28T12:56:19.475 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-28T12:56:19.476 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:56:19.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-28T12:56:19.508 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:56:19.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-28T12:56:19.578 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:56:19.578 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-28T12:56:19.604 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:56:19.605 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-28T12:56:19.671 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:56:19.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-28T12:56:19.740 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:56:19.740 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-28T12:56:19.757 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:56:19.758 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-28T12:56:19.795 INFO:tasks.ceph:Adding keys to all mons... 2024-01-28T12:56:19.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDhTrZly0nyNBAAKZbJKea8bjm26/NXMrWu6g== 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDmTrZlgY3XDRAA65RO68nxcqiEe0gBmqvJUA== 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDeTrZluosaOhAAwKQq2xLuJhO656k394Kkhg== 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDsTrZlLcmwBxAA9P/sMr07/1kEwLKs01dZxA== 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-01-28T12:56:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDwTrZl3N+8FxAAGCrns0p0Uv3br6j8trYDPw== 2024-01-28T12:56:19.826 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-01-28T12:56:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDfTrZl/EqVARAAizrc4JZnJWAmC3l/JkaI8w== 2024-01-28T12:56:19.826 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2024-01-28T12:56:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDfTrZlnCFiBRAAzbWJB9FAnLh+58fSZNeKOA== 2024-01-28T12:56:19.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-28T12:56:19.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-28T12:56:19.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-28T12:56:20.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-28T12:56:20.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-28T12:56:20.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-28T12:56:20.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-28T12:56:20.284 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-28T12:56:20.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-28T12:56:20.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-28T12:56:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.410+0000 7f13f704d9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.410+0000 7f13f704d9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-28T12:56:20.447 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-28T12:56:20.502 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-28T12:56:20.502 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-28T12:56:20.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-28T12:56:20.544 INFO:tasks.ceph.mon.a:Started 2024-01-28T12:56:20.545 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-28T12:56:20.545 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-28T12:56:20.545 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-28T12:56:20.547 INFO:tasks.ceph.mgr.x:Started 2024-01-28T12:56:20.547 INFO:tasks.ceph:Setting crush tunables to default 2024-01-28T12:56:20.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-01-28T12:56:20.622 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-28T12:56:20.623+0000 7fd27204f9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.622 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-28T12:56:20.623+0000 7fd27204f9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.622 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-28T12:56:20.624+0000 7fd27204f9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.638 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:20.639+0000 7f3fdaf6ef00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.650+0000 7f669559e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.656+0000 7f669559e640 1 Processor -- start 2024-01-28T12:56:20.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.656+0000 7f669559e640 1 -- start start 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f669559e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 0x7f669012fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f669559e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669006e7f0 con 0x7f6690098ec0 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f669559e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669006dcd0 con 0x7f669012fad0 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f668effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 0x7f669012fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f66867fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6690098ec0 0x7f66900992d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48372/0 (socket says 172.21.15.31:48372) 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f66867fc640 1 -- 172.21.15.31:0/1898933450 learned_addr learned my addr 172.21.15.31:0/1898933450 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f668effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 0x7f669012fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46280/0 (socket says 172.21.15.31:46280) 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f668effd640 1 -- 172.21.15.31:0/1898933450 >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 msgr2=0x7f669012fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-28T12:56:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f668effd640 1 -- 172.21.15.31:0/1898933450 >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 msgr2=0x7f669012fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-28T12:56:20.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f668effd640 1 --2- 172.21.15.31:0/1898933450 >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 0x7f669012fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-28T12:56:20.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.657+0000 7f668effd640 1 --2- 172.21.15.31:0/1898933450 >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 0x7f669012fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T12:56:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.677+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959503214 0 0) 0x7f669006e7f0 con 0x7f6690098ec0 2024-01-28T12:56:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.677+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f667c005700 con 0x7f6690098ec0 2024-01-28T12:56:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 159707421 0 0) 0x7f667c005700 con 0x7f6690098ec0 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 msgr2=0x7f669012fee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 --2- 172.21.15.31:0/1898933450 >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 0x7f669012fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669012f6a0 con 0x7f6690098ec0 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 254803319 0 0) 0x7f66780015a0 con 0x7f6690098ec0 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 >> v1:172.21.15.31:6789/0 conn(0x7f6690098ec0 legacy=0x7f66900992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 --2- 172.21.15.31:0/1898933450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f667c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669006dcd0 con 0x7f667c005b50 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668effd640 1 --2- 172.21.15.31:0/1898933450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f667c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.678+0000 7f668effd640 1 -- 172.21.15.31:0/1898933450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c005700 con 0x7f667c005b50 2024-01-28T12:56:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.680+0000 7f668effd640 1 --2- 172.21.15.31:0/1898933450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f667c005fa0 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4df8e6b0e82a23d server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.680+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f668000b5c0 con 0x7f667c005b50 2024-01-28T12:56:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.680+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f668000bb00 con 0x7f667c005b50 2024-01-28T12:56:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.680+0000 7f668e7fc640 1 -- 172.21.15.31:0/1898933450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f668000bda0 con 0x7f667c005b50 2024-01-28T12:56:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 -- 172.21.15.31:0/1898933450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 msgr2=0x7f667c005fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 --2- 172.21.15.31:0/1898933450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f667c005fa0 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 -- 172.21.15.31:0/1898933450 shutdown_connections 2024-01-28T12:56:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 --2- 172.21.15.31:0/1898933450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f667c005fa0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 --2- 172.21.15.31:0/1898933450 >> v2:172.21.15.31:3300/0 conn(0x7f669012fad0 0x7f669012fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 -- 172.21.15.31:0/1898933450 >> 172.21.15.31:0/1898933450 conn(0x7f669009f030 msgr2=0x7f669009f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 -- 172.21.15.31:0/1898933450 shutdown_connections 2024-01-28T12:56:20.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 1 -- 172.21.15.31:0/1898933450 wait complete. 2024-01-28T12:56:20.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.681+0000 7f669559e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.680 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:20.680+0000 7f3fdaf6ef00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.681 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:20.681+0000 7f3fdaf6ef00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.682+0000 7f669559e640 1 Processor -- start 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.682+0000 7f669559e640 1 -- start start 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f669559e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f6690129db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f669559e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669012ba20 con 0x7f667c005b50 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f668effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f6690129db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f668effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f6690129db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46292/0 (socket says 172.21.15.31:46292) 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f668effd640 1 -- 172.21.15.31:0/101704940 learned_addr learned my addr 172.21.15.31:0/101704940 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f668effd640 1 -- 172.21.15.31:0/101704940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690097be0 con 0x7f667c005b50 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f668effd640 1 --2- 172.21.15.31:0/101704940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f6690129db0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.683+0000 7f6687fff640 1 -- 172.21.15.31:0/101704940 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6680004250 con 0x7f667c005b50 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.684+0000 7f6687fff640 1 -- 172.21.15.31:0/101704940 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f66800043b0 con 0x7f667c005b50 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.684+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6690097e10 con 0x7f667c005b50 2024-01-28T12:56:20.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.684+0000 7f6687fff640 1 -- 172.21.15.31:0/101704940 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6680015710 con 0x7f667c005b50 2024-01-28T12:56:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.684+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669012cf10 con 0x7f667c005b50 2024-01-28T12:56:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.684+0000 7f6687fff640 1 -- 172.21.15.31:0/101704940 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f6680015c00 con 0x7f667c005b50 2024-01-28T12:56:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.684+0000 7f6687fff640 1 -- 172.21.15.31:0/101704940 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (crc 0 0 0) 0x7f6680016b80 con 0x7f667c005b50 2024-01-28T12:56:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.684+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669012b790 con 0x7f667c005b50 2024-01-28T12:56:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.687+0000 7f6687fff640 1 -- 172.21.15.31:0/101704940 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7f6680003680 con 0x7f667c005b50 2024-01-28T12:56:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.753+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6690075440 con 0x7f667c005b50 2024-01-28T12:56:20.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.754+0000 7f6687fff640 1 -- 172.21.15.31:0/101704940 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (crc 0 0 0) 0x7f6680003820 con 0x7f667c005b50 2024-01-28T12:56:20.752 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-01-28T12:56:20.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.754+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 msgr2=0x7f6690129db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:20.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.754+0000 7f669559e640 1 --2- 172.21.15.31:0/101704940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f6690129db0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.755+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 shutdown_connections 2024-01-28T12:56:20.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.755+0000 7f669559e640 1 --2- 172.21.15.31:0/101704940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c005b50 0x7f6690129db0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.755+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 >> 172.21.15.31:0/101704940 conn(0x7f669009f030 msgr2=0x7f669012d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:20.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.755+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 shutdown_connections 2024-01-28T12:56:20.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.755+0000 7f669559e640 1 -- 172.21.15.31:0/101704940 wait complete. 2024-01-28T12:56:20.763 INFO:tasks.ceph:check_enable_crimson: True 2024-01-28T12:56:20.763 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-01-28T12:56:20.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-01-28T12:56:20.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.865+0000 7f2128aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.870+0000 7f2128aad640 1 Processor -- start 2024-01-28T12:56:20.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.870+0000 7f2128aad640 1 -- start start 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f2128aad640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f212409b520 0x7f2124138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f2128aad640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212409e410 con 0x7f212409aa00 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f2128aad640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212409cdd0 con 0x7f212409b520 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f2121d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f212409b520 0x7f2124138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f212259c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f212409aa00 0x7f212409ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48374/0 (socket says 172.21.15.31:48374) 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f212259c640 1 -- 172.21.15.31:0/3231197591 learned_addr learned my addr 172.21.15.31:0/3231197591 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f2121d9b640 1 -- 172.21.15.31:0/3231197591 >> v1:172.21.15.31:6789/0 conn(0x7f212409aa00 legacy=0x7f212409ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.871+0000 7f2121d9b640 1 -- 172.21.15.31:0/3231197591 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212409d4e0 con 0x7f212409b520 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.872+0000 7f2121d9b640 1 --2- 172.21.15.31:0/3231197591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409b520 0x7f2124138670 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f8b9f3e90f517a server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.872+0000 7f212159a640 1 -- 172.21.15.31:0/3231197591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2118009ce0 con 0x7f212409b520 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.872+0000 7f212159a640 1 -- 172.21.15.31:0/3231197591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2118009e40 con 0x7f212409b520 2024-01-28T12:56:20.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.872+0000 7f212159a640 1 -- 172.21.15.31:0/3231197591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f211800e360 con 0x7f212409b520 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.872+0000 7f2128aad640 1 -- 172.21.15.31:0/3231197591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409b520 msgr2=0x7f2124138670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.872+0000 7f2128aad640 1 --2- 172.21.15.31:0/3231197591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409b520 0x7f2124138670 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.873+0000 7f2128aad640 1 -- 172.21.15.31:0/3231197591 shutdown_connections 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.873+0000 7f2128aad640 1 --2- 172.21.15.31:0/3231197591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409b520 0x7f2124138670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.873+0000 7f2128aad640 1 -- 172.21.15.31:0/3231197591 >> 172.21.15.31:0/3231197591 conn(0x7f21240a3500 msgr2=0x7f21240a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.873+0000 7f2128aad640 1 -- 172.21.15.31:0/3231197591 shutdown_connections 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.873+0000 7f2128aad640 1 -- 172.21.15.31:0/3231197591 wait complete. 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.873+0000 7f2128aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.874+0000 7f2128aad640 1 Processor -- start 2024-01-28T12:56:20.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.874+0000 7f2128aad640 1 -- start start 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.874+0000 7f2128aad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409aa00 0x7f212411c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.874+0000 7f2128aad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124130bc0 con 0x7f212409aa00 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f212259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409aa00 0x7f212411c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f212259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409aa00 0x7f212411c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46306/0 (socket says 172.21.15.31:46306) 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f212259c640 1 -- 172.21.15.31:0/37004831 learned_addr learned my addr 172.21.15.31:0/37004831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f212259c640 1 -- 172.21.15.31:0/37004831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212411dcc0 con 0x7f212409aa00 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f212259c640 1 --2- 172.21.15.31:0/37004831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409aa00 0x7f212411c840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f2112ffd640 1 -- 172.21.15.31:0/37004831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f211c0059d0 con 0x7f212409aa00 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f2112ffd640 1 -- 172.21.15.31:0/37004831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f211c005b30 con 0x7f212409aa00 2024-01-28T12:56:20.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f2112ffd640 1 -- 172.21.15.31:0/37004831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f211c005dd0 con 0x7f212409aa00 2024-01-28T12:56:20.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212411eff0 con 0x7f212409aa00 2024-01-28T12:56:20.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2124120850 con 0x7f212409aa00 2024-01-28T12:56:20.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f2112ffd640 1 -- 172.21.15.31:0/37004831 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f211c007440 con 0x7f212409aa00 2024-01-28T12:56:20.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.875+0000 7f2112ffd640 1 -- 172.21.15.31:0/37004831 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (crc 0 0 0) 0x7f211c00f930 con 0x7f212409aa00 2024-01-28T12:56:20.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.876+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2124072310 con 0x7f212409aa00 2024-01-28T12:56:20.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.878+0000 7f2112ffd640 1 -- 172.21.15.31:0/37004831 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7f211c00fb90 con 0x7f212409aa00 2024-01-28T12:56:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:20.954+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2124075440 con 0x7f212409aa00 2024-01-28T12:56:20.953 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-28T12:56:20.955+0000 7fd2693f1640 -1 WARNING: experimental feature 'crimson' is enabled 2024-01-28T12:56:20.953 INFO:tasks.ceph.mon.a.smithi031.stderr:Please be aware that this feature is experimental, untested, 2024-01-28T12:56:20.953 INFO:tasks.ceph.mon.a.smithi031.stderr:unsupported, and may result in data corruption, data loss, 2024-01-28T12:56:20.953 INFO:tasks.ceph.mon.a.smithi031.stderr:and/or irreparable damage to your cluster. Do not use 2024-01-28T12:56:20.953 INFO:tasks.ceph.mon.a.smithi031.stderr:feature with important data. 2024-01-28T12:56:20.953 INFO:tasks.ceph.mon.a.smithi031.stderr: 2024-01-28T12:56:21.128 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:21.130+0000 7f3fdaf6ef00 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-28T12:56:21.181 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:21.182+0000 7f3fdaf6ef00 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-28T12:56:21.703 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:21.704+0000 7f3fdaf6ef00 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-28T12:56:21.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.754+0000 7f2112ffd640 1 -- 172.21.15.31:0/37004831 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) v1 ==== 99+0+0 (crc 0 0 0) 0x7f211c00fd30 con 0x7f212409aa00 2024-01-28T12:56:21.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.755+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409aa00 msgr2=0x7f212411c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:21.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.755+0000 7f2128aad640 1 --2- 172.21.15.31:0/37004831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409aa00 0x7f212411c840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:21.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.755+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 shutdown_connections 2024-01-28T12:56:21.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.755+0000 7f2128aad640 1 --2- 172.21.15.31:0/37004831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212409aa00 0x7f212411c840 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:21.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.755+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 >> 172.21.15.31:0/37004831 conn(0x7f21240a3500 msgr2=0x7f21241bb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:21.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.756+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 shutdown_connections 2024-01-28T12:56:21.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:21.756+0000 7f2128aad640 1 -- 172.21.15.31:0/37004831 wait complete. 2024-01-28T12:56:21.755 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:21.757+0000 7f3fdaf6ef00 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-28T12:56:21.766 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-28T12:56:21.767 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:56:21.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-28T12:56:21.795 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:56:21.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-28T12:56:21.801 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:21.802+0000 7f3fdaf6ef00 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-28T12:56:21.854 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:21.855+0000 7f3fdaf6ef00 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-28T12:56:21.863 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:56:21.864 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-28T12:56:21.891 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-01-28T12:56:21.891 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-28T12:56:21.898 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:21.899+0000 7f3fdaf6ef00 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-28T12:56:21.924 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new f07c134b-39dd-4d30-b1af-9fe992db79d4 0 2024-01-28T12:56:22.003 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.005+0000 7f3fdaf6ef00 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-28T12:56:22.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.066+0000 7fdc7519e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.072+0000 7fdc7519e640 1 Processor -- start 2024-01-28T12:56:22.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.072+0000 7fdc7519e640 1 -- start start 2024-01-28T12:56:22.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.072+0000 7fdc7519e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc70118220 0x7fdc70118630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.072+0000 7fdc7519e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc70112cf0 con 0x7fdc701121d0 2024-01-28T12:56:22.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.072+0000 7fdc7519e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7004ec60 con 0x7fdc70118220 2024-01-28T12:56:22.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6ed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc70118220 0x7fdc70118630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6ed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdc70118220 0x7fdc70118630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44734/0 (socket says 172.21.15.137:44734) 2024-01-28T12:56:22.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6ed9d640 1 -- 172.21.15.137:0/3752623844 learned_addr learned my addr 172.21.15.137:0/3752623844 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-28T12:56:22.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6e59c640 1 --1- 172.21.15.137:0/3752623844 >> v1:172.21.15.31:6789/0 conn(0x7fdc701121d0 0x7fdc701125e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.137:47146/0 (socket says 172.21.15.137:47146) 2024-01-28T12:56:22.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6ed9d640 1 -- 172.21.15.137:0/3752623844 >> v1:172.21.15.31:6789/0 conn(0x7fdc701121d0 legacy=0x7fdc701125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6ed9d640 1 -- 172.21.15.137:0/3752623844 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc70112e80 con 0x7fdc70118220 2024-01-28T12:56:22.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6ed9d640 1 --2- 172.21.15.137:0/3752623844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc70118220 0x7fdc70118630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156b55641db93287 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6dd9b640 1 -- 172.21.15.137:0/3752623844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdc60009dd0 con 0x7fdc70118220 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6dd9b640 1 -- 172.21.15.137:0/3752623844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdc6000e230 con 0x7fdc70118220 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.073+0000 7fdc6dd9b640 1 -- 172.21.15.137:0/3752623844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdc6000e4d0 con 0x7fdc70118220 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 1 -- 172.21.15.137:0/3752623844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc70118220 msgr2=0x7fdc70118630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 1 --2- 172.21.15.137:0/3752623844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc70118220 0x7fdc70118630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 1 -- 172.21.15.137:0/3752623844 shutdown_connections 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 1 --2- 172.21.15.137:0/3752623844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc70118220 0x7fdc70118630 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 1 -- 172.21.15.137:0/3752623844 >> 172.21.15.137:0/3752623844 conn(0x7fdc7010dee0 msgr2=0x7fdc70110300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 1 -- 172.21.15.137:0/3752623844 shutdown_connections 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 1 -- 172.21.15.137:0/3752623844 wait complete. 2024-01-28T12:56:22.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.074+0000 7fdc7519e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.075+0000 7fdc7519e640 1 Processor -- start 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.075+0000 7fdc7519e640 1 -- start start 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.075+0000 7fdc7519e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701121d0 0x7fdc701a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.075+0000 7fdc7519e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701a73c0 con 0x7fdc701121d0 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.076+0000 7fdc6ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701121d0 0x7fdc701a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.076+0000 7fdc6ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701121d0 0x7fdc701a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44748/0 (socket says 172.21.15.137:44748) 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.076+0000 7fdc6ed9d640 1 -- 172.21.15.137:0/2981360735 learned_addr learned my addr 172.21.15.137:0/2981360735 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-28T12:56:22.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.076+0000 7fdc6ed9d640 1 -- 172.21.15.137:0/2981360735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc701a7500 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.076+0000 7fdc6ed9d640 1 --2- 172.21.15.137:0/2981360735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701121d0 0x7fdc701a6df0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.076+0000 7fdc577fe640 1 -- 172.21.15.137:0/2981360735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdc60014540 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.077+0000 7fdc577fe640 1 -- 172.21.15.137:0/2981360735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdc600146a0 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.077+0000 7fdc577fe640 1 -- 172.21.15.137:0/2981360735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdc60014940 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.077+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc701a7750 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.077+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc701ab600 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.077+0000 7fdc577fe640 1 -- 172.21.15.137:0/2981360735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7fdc60013400 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.077+0000 7fdc577fe640 1 -- 172.21.15.137:0/2981360735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 747+0+0 (crc 0 0 0) 0x7fdc60019040 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.077+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc700675d0 con 0x7fdc701121d0 2024-01-28T12:56:22.077 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.079+0000 7f3fdaf6ef00 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-28T12:56:22.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.080+0000 7fdc577fe640 1 -- 172.21.15.137:0/2981360735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7fdc60013890 con 0x7fdc701121d0 2024-01-28T12:56:22.124 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.125+0000 7f3fdaf6ef00 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-28T12:56:22.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.150+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f07c134b-39dd-4d30-b1af-9fe992db79d4", "id": 0} v 0) v1 -- 0x7fdc70118220 con 0x7fdc701121d0 2024-01-28T12:56:22.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.151+0000 7fdc577fe640 1 -- 172.21.15.137:0/2981360735 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f07c134b-39dd-4d30-b1af-9fe992db79d4", "id": 0}]=0 v4) v1 ==== 112+0+2 (crc 0 0 0) 0x7fdc60013a30 con 0x7fdc701121d0 2024-01-28T12:56:22.150 INFO:teuthology.orchestra.run.smithi137.stdout:0 2024-01-28T12:56:22.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.152+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701121d0 msgr2=0x7fdc701a6df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.152+0000 7fdc7519e640 1 --2- 172.21.15.137:0/2981360735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701121d0 0x7fdc701a6df0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.152+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 shutdown_connections 2024-01-28T12:56:22.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.152+0000 7fdc7519e640 1 --2- 172.21.15.137:0/2981360735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701121d0 0x7fdc701a6df0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.152+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 >> 172.21.15.137:0/2981360735 conn(0x7fdc7010dee0 msgr2=0x7fdc70113ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.152+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 shutdown_connections 2024-01-28T12:56:22.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.152+0000 7fdc7519e640 1 -- 172.21.15.137:0/2981360735 wait complete. 2024-01-28T12:56:22.161 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 4d58f6cd-02f5-44e3-9b51-9e99f3720656 1 2024-01-28T12:56:22.177 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.179+0000 7f3fdaf6ef00 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-28T12:56:22.227 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.228+0000 7f3fdaf6ef00 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-28T12:56:22.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.279+0000 7f422851e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f422851e640 1 Processor -- start 2024-01-28T12:56:22.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f422851e640 1 -- start start 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f422851e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f422011b000 0x7f422011d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f422851e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422011d9f0 con 0x7f4220118220 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f422851e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422004ec60 con 0x7f422011b000 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f4225ab9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f422011b000 0x7f422011d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f4225ab9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f422011b000 0x7f422011d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44756/0 (socket says 172.21.15.137:44756) 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f42262ba640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4220118220 0x7f4220118630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.137:47160/0 (socket says 172.21.15.137:47160) 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.285+0000 7f4225ab9640 1 -- 172.21.15.137:0/2453948931 learned_addr learned my addr 172.21.15.137:0/2453948931 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-28T12:56:22.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.286+0000 7f4225ab9640 1 -- 172.21.15.137:0/2453948931 >> v1:172.21.15.31:6789/0 conn(0x7f4220118220 legacy=0x7f4220118630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.286+0000 7f4225ab9640 1 -- 172.21.15.137:0/2453948931 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422011ebc0 con 0x7f422011b000 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.286+0000 7f4225ab9640 1 --2- 172.21.15.137:0/2453948931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f422011b000 0x7f422011d420 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325b637b525576c server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.286+0000 7f42252b8640 1 -- 172.21.15.137:0/2453948931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4210009dd0 con 0x7f422011b000 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.286+0000 7f42252b8640 1 -- 172.21.15.137:0/2453948931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f421000e230 con 0x7f422011b000 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.286+0000 7f42252b8640 1 -- 172.21.15.137:0/2453948931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f421000e4d0 con 0x7f422011b000 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 1 -- 172.21.15.137:0/2453948931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f422011b000 msgr2=0x7f422011d420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 1 --2- 172.21.15.137:0/2453948931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f422011b000 0x7f422011d420 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 1 -- 172.21.15.137:0/2453948931 shutdown_connections 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 1 --2- 172.21.15.137:0/2453948931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f422011b000 0x7f422011d420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 1 -- 172.21.15.137:0/2453948931 >> 172.21.15.137:0/2453948931 conn(0x7f422010dee0 msgr2=0x7f4220110300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 1 -- 172.21.15.137:0/2453948931 shutdown_connections 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 1 -- 172.21.15.137:0/2453948931 wait complete. 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.287+0000 7f422851e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.288+0000 7f422851e640 1 Processor -- start 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.288+0000 7f422851e640 1 -- start start 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.288+0000 7f422851e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4220118220 0x7f42201a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.288+0000 7f422851e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42201a7390 con 0x7f4220118220 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.288+0000 7f42262ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4220118220 0x7f42201a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.288+0000 7f42262ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4220118220 0x7f42201a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44760/0 (socket says 172.21.15.137:44760) 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.288+0000 7f42262ba640 1 -- 172.21.15.137:0/2463596930 learned_addr learned my addr 172.21.15.137:0/2463596930 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-28T12:56:22.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f42262ba640 1 -- 172.21.15.137:0/2463596930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42201a74d0 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f42262ba640 1 --2- 172.21.15.137:0/2463596930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4220118220 0x7f42201a6dc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f4216ffd640 1 -- 172.21.15.137:0/2463596930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f421c0071e0 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f4216ffd640 1 -- 172.21.15.137:0/2463596930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f421c007340 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42201a7780 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f4216ffd640 1 -- 172.21.15.137:0/2463596930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f421c0075e0 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42201ab5d0 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f4216ffd640 1 -- 172.21.15.137:0/2463596930 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f421c005b70 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.289+0000 7f4216ffd640 1 -- 172.21.15.137:0/2463596930 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 862+0+0 (crc 0 0 0) 0x7f421c00fa00 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.290+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42201a7970 con 0x7f4220118220 2024-01-28T12:56:22.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.292+0000 7f4216ffd640 1 -- 172.21.15.137:0/2463596930 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7f421c00fbc0 con 0x7f4220118220 2024-01-28T12:56:22.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.363+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4d58f6cd-02f5-44e3-9b51-9e99f3720656", "id": 1} v 0) v1 -- 0x7f422011b000 con 0x7f4220118220 2024-01-28T12:56:22.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.365+0000 7f4216ffd640 1 -- 172.21.15.137:0/2463596930 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4d58f6cd-02f5-44e3-9b51-9e99f3720656", "id": 1}]=0 v5) v1 ==== 112+0+2 (crc 0 0 0) 0x7f421c00fde0 con 0x7f4220118220 2024-01-28T12:56:22.363 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-01-28T12:56:22.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.365+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4220118220 msgr2=0x7f42201a6dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.365+0000 7f422851e640 1 --2- 172.21.15.137:0/2463596930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4220118220 0x7f42201a6dc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.365+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 shutdown_connections 2024-01-28T12:56:22.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.365+0000 7f422851e640 1 --2- 172.21.15.137:0/2463596930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4220118220 0x7f42201a6dc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.366+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 >> 172.21.15.137:0/2463596930 conn(0x7f422010dee0 msgr2=0x7f4220110300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.366+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 shutdown_connections 2024-01-28T12:56:22.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.366+0000 7f422851e640 1 -- 172.21.15.137:0/2463596930 wait complete. 2024-01-28T12:56:22.374 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 05ce0150-eebc-405f-8fb8-fe1ef00b7cd2 2 2024-01-28T12:56:22.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.477+0000 7f6063060640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.483+0000 7f6063060640 1 Processor -- start 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.483+0000 7f6063060640 1 -- start start 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f6063060640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f605c118160 0x7f605c118570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f6063060640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c112be0 con 0x7f605c1120c0 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f6063060640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605c118d50 con 0x7f605c118160 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f6060dfc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f605c118160 0x7f605c118570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f605bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f605c1120c0 0x7f605c1124d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.137:47174/0 (socket says 172.21.15.137:47174) 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f605bfff640 1 -- 172.21.15.137:0/59919276 learned_addr learned my addr 172.21.15.137:0/59919276 (peer_addr_for_me v1:172.21.15.137:0/0) 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f6060dfc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f605c118160 0x7f605c118570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44770/0 (socket says 172.21.15.137:44770) 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.484+0000 7f6060dfc640 1 -- 172.21.15.137:0/59919276 >> v1:172.21.15.31:6789/0 conn(0x7f605c1120c0 legacy=0x7f605c1124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.485+0000 7f605b7fe640 1 -- 172.21.15.137:0/59919276 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456029621 0 0) 0x7f605c112be0 con 0x7f605c1120c0 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.485+0000 7f6060dfc640 1 -- 172.21.15.137:0/59919276 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c112da0 con 0x7f605c118160 2024-01-28T12:56:22.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.485+0000 7f6060dfc640 1 --2- 172.21.15.137:0/59919276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c118160 0x7f605c118570 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9e0dde52c99d3e server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.485+0000 7f605b7fe640 1 -- 172.21.15.137:0/59919276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6050009ce0 con 0x7f605c118160 2024-01-28T12:56:22.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.485+0000 7f605b7fe640 1 -- 172.21.15.137:0/59919276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6050009e40 con 0x7f605c118160 2024-01-28T12:56:22.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.485+0000 7f605b7fe640 1 -- 172.21.15.137:0/59919276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f605000e360 con 0x7f605c118160 2024-01-28T12:56:22.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 1 -- 172.21.15.137:0/59919276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c118160 msgr2=0x7f605c118570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 1 --2- 172.21.15.137:0/59919276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c118160 0x7f605c118570 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 1 -- 172.21.15.137:0/59919276 shutdown_connections 2024-01-28T12:56:22.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 1 --2- 172.21.15.137:0/59919276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c118160 0x7f605c118570 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 1 -- 172.21.15.137:0/59919276 >> 172.21.15.137:0/59919276 conn(0x7f605c10dee0 msgr2=0x7f605c110300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 1 -- 172.21.15.137:0/59919276 shutdown_connections 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 1 -- 172.21.15.137:0/59919276 wait complete. 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.486+0000 7f6063060640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.487+0000 7f6063060640 1 Processor -- start 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.487+0000 7f6063060640 1 -- start start 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.487+0000 7f6063060640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c1120c0 0x7f605c1a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.487+0000 7f6063060640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c1a7200 con 0x7f605c1120c0 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.487+0000 7f6060dfc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c1120c0 0x7f605c1a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.487+0000 7f6060dfc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c1120c0 0x7f605c1a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44780/0 (socket says 172.21.15.137:44780) 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.487+0000 7f6060dfc640 1 -- 172.21.15.137:0/53266050 learned_addr learned my addr 172.21.15.137:0/53266050 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f6060dfc640 1 -- 172.21.15.137:0/53266050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c1a7340 con 0x7f605c1120c0 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f6060dfc640 1 --2- 172.21.15.137:0/53266050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c1120c0 0x7f605c1a6cc0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f60597fa640 1 -- 172.21.15.137:0/53266050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6050014540 con 0x7f605c1120c0 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f60597fa640 1 -- 172.21.15.137:0/53266050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f60500146a0 con 0x7f605c1120c0 2024-01-28T12:56:22.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c1a7590 con 0x7f605c1120c0 2024-01-28T12:56:22.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f60597fa640 1 -- 172.21.15.137:0/53266050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6050014940 con 0x7f605c1120c0 2024-01-28T12:56:22.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c1ab3d0 con 0x7f605c1120c0 2024-01-28T12:56:22.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f60597fa640 1 -- 172.21.15.137:0/53266050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f6050013400 con 0x7f605c1120c0 2024-01-28T12:56:22.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.488+0000 7f60597fa640 1 -- 172.21.15.137:0/53266050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 977+0+0 (crc 0 0 0) 0x7f6050019040 con 0x7f605c1120c0 2024-01-28T12:56:22.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.489+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c0675d0 con 0x7f605c1120c0 2024-01-28T12:56:22.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.491+0000 7f60597fa640 1 -- 172.21.15.137:0/53266050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7f60500125c0 con 0x7f605c1120c0 2024-01-28T12:56:22.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.563+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "05ce0150-eebc-405f-8fb8-fe1ef00b7cd2", "id": 2} v 0) v1 -- 0x7f605c118190 con 0x7f605c1120c0 2024-01-28T12:56:22.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.564+0000 7f60597fa640 1 -- 172.21.15.137:0/53266050 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "05ce0150-eebc-405f-8fb8-fe1ef00b7cd2", "id": 2}]=0 v6) v1 ==== 112+0+2 (crc 0 0 0) 0x7f6050012760 con 0x7f605c1120c0 2024-01-28T12:56:22.563 INFO:teuthology.orchestra.run.smithi137.stdout:2 2024-01-28T12:56:22.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.565+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c1120c0 msgr2=0x7f605c1a6cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.565+0000 7f6063060640 1 --2- 172.21.15.137:0/53266050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c1120c0 0x7f605c1a6cc0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.565+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 shutdown_connections 2024-01-28T12:56:22.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.565+0000 7f6063060640 1 --2- 172.21.15.137:0/53266050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605c1120c0 0x7f605c1a6cc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.565+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 >> 172.21.15.137:0/53266050 conn(0x7f605c10dee0 msgr2=0x7f605c113bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.565+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 shutdown_connections 2024-01-28T12:56:22.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.565+0000 7f6063060640 1 -- 172.21.15.137:0/53266050 wait complete. 2024-01-28T12:56:22.574 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new bd000973-288c-4ebb-a3f5-871638c8e946 3 2024-01-28T12:56:22.629 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.630+0000 7f3fdaf6ef00 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-28T12:56:22.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.676+0000 7f868a42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.681+0000 7f868a42d640 1 Processor -- start 2024-01-28T12:56:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.681+0000 7f868a42d640 1 -- start start 2024-01-28T12:56:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f868a42d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8684118220 0x7f8684118630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f868a42d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8684112cf0 con 0x7f86841121d0 2024-01-28T12:56:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f868a42d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868404ec60 con 0x7f8684118220 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f8683fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8684118220 0x7f8684118630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f8683fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8684118220 0x7f8684118630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44792/0 (socket says 172.21.15.137:44792) 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f8683fff640 1 -- 172.21.15.137:0/1957408799 learned_addr learned my addr 172.21.15.137:0/1957408799 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f8683fff640 1 -- 172.21.15.137:0/1957408799 >> v1:172.21.15.31:6789/0 conn(0x7f86841121d0 legacy=0x7f86841125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.682+0000 7f8683fff640 1 -- 172.21.15.137:0/1957408799 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684112e80 con 0x7f8684118220 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.683+0000 7f8683fff640 1 --2- 172.21.15.137:0/1957408799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8684118220 0x7f8684118630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657e08d0d4f2ee1e server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.683+0000 7f8682ffd640 1 -- 172.21.15.137:0/1957408799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8674009dd0 con 0x7f8684118220 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.683+0000 7f8682ffd640 1 -- 172.21.15.137:0/1957408799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f867400e230 con 0x7f8684118220 2024-01-28T12:56:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.683+0000 7f8682ffd640 1 -- 172.21.15.137:0/1957408799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f867400e4d0 con 0x7f8684118220 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.683+0000 7f868a42d640 1 -- 172.21.15.137:0/1957408799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8684118220 msgr2=0x7f8684118630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.683+0000 7f868a42d640 1 --2- 172.21.15.137:0/1957408799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8684118220 0x7f8684118630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 1 -- 172.21.15.137:0/1957408799 shutdown_connections 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 1 --2- 172.21.15.137:0/1957408799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8684118220 0x7f8684118630 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 1 -- 172.21.15.137:0/1957408799 >> 172.21.15.137:0/1957408799 conn(0x7f868410dee0 msgr2=0x7f8684110300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 1 -- 172.21.15.137:0/1957408799 shutdown_connections 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 1 -- 172.21.15.137:0/1957408799 wait complete. 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 1 Processor -- start 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.684+0000 7f868a42d640 1 -- start start 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.685+0000 7f868a42d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86841121d0 0x7f86841a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.685+0000 7f868a42d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841a73c0 con 0x7f86841121d0 2024-01-28T12:56:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.685+0000 7f8683fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86841121d0 0x7f86841a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.685+0000 7f8683fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86841121d0 0x7f86841a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.137:44808/0 (socket says 172.21.15.137:44808) 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.685+0000 7f8683fff640 1 -- 172.21.15.137:0/1556018104 learned_addr learned my addr 172.21.15.137:0/1556018104 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.685+0000 7f8683fff640 1 -- 172.21.15.137:0/1556018104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86841a7500 con 0x7f86841121d0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f8683fff640 1 --2- 172.21.15.137:0/1556018104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86841121d0 0x7f86841a6df0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f8680ff9640 1 -- 172.21.15.137:0/1556018104 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8674014540 con 0x7f86841121d0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f8680ff9640 1 -- 172.21.15.137:0/1556018104 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f86740146a0 con 0x7f86841121d0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86841a7750 con 0x7f86841121d0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f8680ff9640 1 -- 172.21.15.137:0/1556018104 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8674014940 con 0x7f86841121d0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86841ab600 con 0x7f86841121d0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f8680ff9640 1 -- 172.21.15.137:0/1556018104 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f8674013400 con 0x7f86841121d0 2024-01-28T12:56:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.686+0000 7f8680ff9640 1 -- 172.21.15.137:0/1556018104 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1092+0+0 (crc 0 0 0) 0x7f8674019040 con 0x7f86841121d0 2024-01-28T12:56:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.687+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86840675d0 con 0x7f86841121d0 2024-01-28T12:56:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.689+0000 7f8680ff9640 1 -- 172.21.15.137:0/1556018104 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7f86740125c0 con 0x7f86841121d0 2024-01-28T12:56:22.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.761+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bd000973-288c-4ebb-a3f5-871638c8e946", "id": 3} v 0) v1 -- 0x7f8684118220 con 0x7f86841121d0 2024-01-28T12:56:22.761 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.762+0000 7f3fdaf6ef00 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-28T12:56:22.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.765+0000 7f8680ff9640 1 -- 172.21.15.137:0/1556018104 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd000973-288c-4ebb-a3f5-871638c8e946", "id": 3}]=0 v7) v1 ==== 112+0+2 (crc 0 0 0) 0x7f8674012760 con 0x7f86841121d0 2024-01-28T12:56:22.763 INFO:teuthology.orchestra.run.smithi137.stdout:3 2024-01-28T12:56:22.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.766+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86841121d0 msgr2=0x7f86841a6df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:22.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.766+0000 7f868a42d640 1 --2- 172.21.15.137:0/1556018104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86841121d0 0x7f86841a6df0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.766+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 shutdown_connections 2024-01-28T12:56:22.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.766+0000 7f868a42d640 1 --2- 172.21.15.137:0/1556018104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86841121d0 0x7f86841a6df0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:22.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.766+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 >> 172.21.15.137:0/1556018104 conn(0x7f868410dee0 msgr2=0x7f8684113ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:22.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.766+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 shutdown_connections 2024-01-28T12:56:22.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-01-28T12:56:22.766+0000 7f868a42d640 1 -- 172.21.15.137:0/1556018104 wait complete. 2024-01-28T12:56:22.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-01-28T12:56:22.811 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-28T12:56:22.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-01-28T12:56:22.853 INFO:tasks.ceph.osd.0:Started 2024-01-28T12:56:22.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-01-28T12:56:22.887 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-28T12:56:22.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-01-28T12:56:22.890 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.892+0000 7f3fdaf6ef00 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-28T12:56:22.929 INFO:tasks.ceph.osd.1:Started 2024-01-28T12:56:22.929 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-01-28T12:56:22.938 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:22.939+0000 7f3fdaf6ef00 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-28T12:56:22.962 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-28T12:56:22.963 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-01-28T12:56:23.005 INFO:tasks.ceph.osd.2:Started 2024-01-28T12:56:23.005 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-01-28T12:56:23.036 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-28T12:56:23.037 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-01-28T12:56:23.044 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:23.045+0000 7f3fdaf6ef00 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-28T12:56:23.046 INFO:tasks.ceph.osd.3:Started 2024-01-28T12:56:23.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-28T12:56:23.152 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:23.154+0000 7f3fdaf6ef00 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-28T12:56:23.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.170+0000 7f5d7766d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:23.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.175+0000 7f5d7766d640 1 Processor -- start 2024-01-28T12:56:23.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.176+0000 7f5d7766d640 1 -- start start 2024-01-28T12:56:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.176+0000 7f5d7766d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d7009cbf0 0x7f5d70133140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.176+0000 7f5d7766d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7009c7b0 con 0x7f5d70133680 2024-01-28T12:56:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.176+0000 7f5d7766d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7009d7b0 con 0x7f5d7009cbf0 2024-01-28T12:56:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d75409640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d7009cbf0 0x7f5d70133140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d74c08640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5d70133680 0x7f5d7012fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48378/0 (socket says 172.21.15.31:48378) 2024-01-28T12:56:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d74c08640 1 -- 172.21.15.31:0/840446962 learned_addr learned my addr 172.21.15.31:0/840446962 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:23.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d75409640 1 -- 172.21.15.31:0/840446962 >> v1:172.21.15.31:6789/0 conn(0x7f5d70133680 legacy=0x7f5d7012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:23.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d75409640 1 -- 172.21.15.31:0/840446962 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7009e3c0 con 0x7f5d7009cbf0 2024-01-28T12:56:23.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d67fff640 1 -- 172.21.15.31:0/840446962 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019936074 0 0) 0x7f5d7009c7b0 con 0x7f5d70133680 2024-01-28T12:56:23.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d75409640 1 --2- 172.21.15.31:0/840446962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d70133140 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6a3c407d8d2b37 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:23.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d67fff640 1 -- 172.21.15.31:0/840446962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d6c009ce0 con 0x7f5d7009cbf0 2024-01-28T12:56:23.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d67fff640 1 -- 172.21.15.31:0/840446962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5d6c009e40 con 0x7f5d7009cbf0 2024-01-28T12:56:23.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.177+0000 7f5d67fff640 1 -- 172.21.15.31:0/840446962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d6c00e360 con 0x7f5d7009cbf0 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.178+0000 7f5d7766d640 1 -- 172.21.15.31:0/840446962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 msgr2=0x7f5d70133140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.178+0000 7f5d7766d640 1 --2- 172.21.15.31:0/840446962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d70133140 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.178+0000 7f5d7766d640 1 -- 172.21.15.31:0/840446962 shutdown_connections 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.178+0000 7f5d7766d640 1 --2- 172.21.15.31:0/840446962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d70133140 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.178+0000 7f5d7766d640 1 -- 172.21.15.31:0/840446962 >> 172.21.15.31:0/840446962 conn(0x7f5d700a5ff0 msgr2=0x7f5d700989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.178+0000 7f5d7766d640 1 -- 172.21.15.31:0/840446962 shutdown_connections 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.178+0000 7f5d7766d640 1 -- 172.21.15.31:0/840446962 wait complete. 2024-01-28T12:56:23.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.179+0000 7f5d7766d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:23.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.180+0000 7f5d7766d640 1 Processor -- start 2024-01-28T12:56:23.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.180+0000 7f5d7766d640 1 -- start start 2024-01-28T12:56:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.180+0000 7f5d7766d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d7011cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.180+0000 7f5d7766d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7011f530 con 0x7f5d7009cbf0 2024-01-28T12:56:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d75409640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d7011cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d75409640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d7011cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46322/0 (socket says 172.21.15.31:46322) 2024-01-28T12:56:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d75409640 1 -- 172.21.15.31:0/4147590516 learned_addr learned my addr 172.21.15.31:0/4147590516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:23.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d75409640 1 -- 172.21.15.31:0/4147590516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7011fdc0 con 0x7f5d7009cbf0 2024-01-28T12:56:23.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d75409640 1 --2- 172.21.15.31:0/4147590516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d7011cb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:23.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d65ffb640 1 -- 172.21.15.31:0/4147590516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d6c014540 con 0x7f5d7009cbf0 2024-01-28T12:56:23.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d65ffb640 1 -- 172.21.15.31:0/4147590516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5d6c0146a0 con 0x7f5d7009cbf0 2024-01-28T12:56:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d65ffb640 1 -- 172.21.15.31:0/4147590516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d6c014940 con 0x7f5d7009cbf0 2024-01-28T12:56:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.181+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7011f720 con 0x7f5d7009cbf0 2024-01-28T12:56:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.182+0000 7f5d65ffb640 1 -- 172.21.15.31:0/4147590516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f5d6c011380 con 0x7f5d7009cbf0 2024-01-28T12:56:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.182+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d70120ae0 con 0x7f5d7009cbf0 2024-01-28T12:56:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.182+0000 7f5d65ffb640 1 -- 172.21.15.31:0/4147590516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f5d6c019040 con 0x7f5d7009cbf0 2024-01-28T12:56:23.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.183+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7011f280 con 0x7f5d7009cbf0 2024-01-28T12:56:23.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.186+0000 7f5d65ffb640 1 -- 172.21.15.31:0/4147590516 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7f5d6c010310 con 0x7f5d7009cbf0 2024-01-28T12:56:23.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.280+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d70075440 con 0x7f5d7009cbf0 2024-01-28T12:56:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.281+0000 7f5d65ffb640 1 -- 172.21.15.31:0/4147590516 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4152 (crc 0 0 0) 0x7f5d6c0118b0 con 0x7f5d7009cbf0 2024-01-28T12:56:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:56:23.279 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"fsid":"68a362e5-2887-44cb-94c6-4a7b059316bf","created":"2024-01-28T12:56:20.671303+0000","modified":"2024-01-28T12:56:22.763590+0000","last_up_change":"0.000000","last_in_change":"2024-01-28T12:56:22.763590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"f07c134b-39dd-4d30-b1af-9fe992db79d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d58f6cd-02f5-44e3-9b51-9e99f3720656","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"05ce0150-eebc-405f-8fb8-fe1ef00b7cd2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd000973-288c-4ebb-a3f5-871638c8e946","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-28T12:56:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.282+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 msgr2=0x7f5d7011cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.282+0000 7f5d7766d640 1 --2- 172.21.15.31:0/4147590516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d7011cb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.282+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 shutdown_connections 2024-01-28T12:56:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.282+0000 7f5d7766d640 1 --2- 172.21.15.31:0/4147590516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7009cbf0 0x7f5d7011cb00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.282+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 >> 172.21.15.31:0/4147590516 conn(0x7f5d700a5ff0 msgr2=0x7f5d70130f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.282+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 shutdown_connections 2024-01-28T12:56:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:23.282+0000 7f5d7766d640 1 -- 172.21.15.31:0/4147590516 wait complete. 2024-01-28T12:56:23.293 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-28T12:56:23.293 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-28T12:56:23.318 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:23.320+0000 7f3fdaf6ef00 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-28T12:56:23.369 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:23.370+0000 7f3fdaf6ef00 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-28T12:56:23.528 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:23.529+0000 7f3fdaf6ef00 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-28T12:56:23.738 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-28T12:56:23.838 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.836+0000 7f34ee19e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:23.838 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.840+0000 7f34ee19e640 1 Processor -- start 2024-01-28T12:56:23.838 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.840+0000 7f34ee19e640 1 -- start start 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.841+0000 7f34ee19e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34e809b520 0x7f34e8138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.841+0000 7f34ee19e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e809cb60 con 0x7f34e809aa00 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.841+0000 7f34ee19e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e809cdc0 con 0x7f34e809b520 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.841+0000 7f34e6ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34e809b520 0x7f34e8138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.841+0000 7f34e77fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f34e809aa00 0x7f34e809ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48394/0 (socket says 172.21.15.31:48394) 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.841+0000 7f34e77fe640 1 -- 172.21.15.31:0/4274938268 learned_addr learned my addr 172.21.15.31:0/4274938268 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.842+0000 7f34e6ffd640 1 -- 172.21.15.31:0/4274938268 >> v1:172.21.15.31:6789/0 conn(0x7f34e809aa00 legacy=0x7f34e809ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:23.840 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.842+0000 7f34e6ffd640 1 -- 172.21.15.31:0/4274938268 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e80a2f10 con 0x7f34e809b520 2024-01-28T12:56:23.841 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.842+0000 7f34e6ffd640 1 --2- 172.21.15.31:0/4274938268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809b520 0x7f34e8138670 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8843f983ac56c1d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:23.841 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.842+0000 7f34e67fc640 1 -- 172.21.15.31:0/4274938268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f34dc009ce0 con 0x7f34e809b520 2024-01-28T12:56:23.841 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.842+0000 7f34e67fc640 1 -- 172.21.15.31:0/4274938268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f34dc009e40 con 0x7f34e809b520 2024-01-28T12:56:23.841 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.842+0000 7f34e67fc640 1 -- 172.21.15.31:0/4274938268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f34dc00e360 con 0x7f34e809b520 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 1 -- 172.21.15.31:0/4274938268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809b520 msgr2=0x7f34e8138670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 1 --2- 172.21.15.31:0/4274938268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809b520 0x7f34e8138670 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 1 -- 172.21.15.31:0/4274938268 shutdown_connections 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 1 --2- 172.21.15.31:0/4274938268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809b520 0x7f34e8138670 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 1 -- 172.21.15.31:0/4274938268 >> 172.21.15.31:0/4274938268 conn(0x7f34e80a3500 msgr2=0x7f34e80a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 1 -- 172.21.15.31:0/4274938268 shutdown_connections 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 1 -- 172.21.15.31:0/4274938268 wait complete. 2024-01-28T12:56:23.842 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.843+0000 7f34ee19e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:23.843 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.845+0000 7f34ee19e640 1 Processor -- start 2024-01-28T12:56:23.843 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.845+0000 7f34ee19e640 1 -- start start 2024-01-28T12:56:23.845 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34ee19e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809aa00 0x7f34e811dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:23.845 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34ee19e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e81209d0 con 0x7f34e809aa00 2024-01-28T12:56:23.845 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809aa00 0x7f34e811dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:23.845 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809aa00 0x7f34e811dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46334/0 (socket says 172.21.15.31:46334) 2024-01-28T12:56:23.845 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34e77fe640 1 -- 172.21.15.31:0/2815726287 learned_addr learned my addr 172.21.15.31:0/2815726287 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34e77fe640 1 -- 172.21.15.31:0/2815726287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e8120fa0 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34e77fe640 1 --2- 172.21.15.31:0/2815726287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809aa00 0x7f34e811dc50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.846+0000 7f34cffff640 1 -- 172.21.15.31:0/2815726287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f34d80059d0 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.847+0000 7f34cffff640 1 -- 172.21.15.31:0/2815726287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f34d8005b30 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.847+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e811f7d0 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.847+0000 7f34cffff640 1 -- 172.21.15.31:0/2815726287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f34d8005dd0 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.847+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e8120730 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.847+0000 7f34cffff640 1 -- 172.21.15.31:0/2815726287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f34d8007440 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.847+0000 7f34cffff640 1 -- 172.21.15.31:0/2815726287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f34d800fb10 con 0x7f34e809aa00 2024-01-28T12:56:23.846 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.848+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e8062560 con 0x7f34e809aa00 2024-01-28T12:56:23.847 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.850+0000 7f34cffff640 1 -- 172.21.15.31:0/2815726287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (crc 0 0 0) 0x7f34d800fd70 con 0x7f34e809aa00 2024-01-28T12:56:23.910 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.912+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34e8133d40 con 0x7f34e809aa00 2024-01-28T12:56:23.911 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.913+0000 7f34cffff640 1 -- 172.21.15.31:0/2815726287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4152 (crc 0 0 0) 0x7f34d800f5f0 con 0x7f34e809aa00 2024-01-28T12:56:23.911 INFO:teuthology.misc.health.smithi031.stdout: 2024-01-28T12:56:23.911 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":7,"fsid":"68a362e5-2887-44cb-94c6-4a7b059316bf","created":"2024-01-28T12:56:20.671303+0000","modified":"2024-01-28T12:56:22.763590+0000","last_up_change":"0.000000","last_in_change":"2024-01-28T12:56:22.763590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"f07c134b-39dd-4d30-b1af-9fe992db79d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d58f6cd-02f5-44e3-9b51-9e99f3720656","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"05ce0150-eebc-405f-8fb8-fe1ef00b7cd2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd000973-288c-4ebb-a3f5-871638c8e946","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-28T12:56:23.912 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.914+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809aa00 msgr2=0x7f34e811dc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:23.912 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.914+0000 7f34ee19e640 1 --2- 172.21.15.31:0/2815726287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809aa00 0x7f34e811dc50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.912 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.914+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 shutdown_connections 2024-01-28T12:56:23.912 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.914+0000 7f34ee19e640 1 --2- 172.21.15.31:0/2815726287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e809aa00 0x7f34e811dc50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:23.912 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.914+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 >> 172.21.15.31:0/2815726287 conn(0x7f34e80a3500 msgr2=0x7f34e80989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:23.913 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.914+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 shutdown_connections 2024-01-28T12:56:23.913 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:23.914+0000 7f34ee19e640 1 -- 172.21.15.31:0/2815726287 wait complete. 2024-01-28T12:56:23.921 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-01-28T12:56:24.384 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:24.385+0000 7f3fdaf6ef00 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-28T12:56:24.658 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:24.659+0000 7f3fdaf6ef00 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-28T12:56:24.718 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:24.719+0000 7f3fdaf6ef00 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-28T12:56:24.824 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:24.826+0000 7f3fdaf6ef00 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-28T12:56:25.846 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T12:56:25.848+0000 7f3f7acd5640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-01-28T12:56:30.325 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-28T12:56:30.422 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.420+0000 7fb749b5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:30.422 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.424+0000 7fb749b5e640 1 Processor -- start 2024-01-28T12:56:30.422 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.424+0000 7fb749b5e640 1 -- start start 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb749b5e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb74409b520 0x7fb744138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb749b5e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74409cb60 con 0x7fb74409aa00 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb749b5e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74409cdc0 con 0x7fb74409b520 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb7437fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb74409aa00 0x7fb74409ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60836/0 (socket says 172.21.15.31:60836) 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb7437fe640 1 -- 172.21.15.31:0/965012997 learned_addr learned my addr 172.21.15.31:0/965012997 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb742ffd640 1 --2- 172.21.15.31:0/965012997 >> v2:172.21.15.31:3300/0 conn(0x7fb74409b520 0x7fb744138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb7427fc640 1 -- 172.21.15.31:0/965012997 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101084972 0 0) 0x7fb74409cb60 con 0x7fb74409aa00 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.425+0000 7fb7427fc640 1 -- 172.21.15.31:0/965012997 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb72c0034f0 con 0x7fb74409aa00 2024-01-28T12:56:30.425 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb742ffd640 1 -- 172.21.15.31:0/965012997 >> v1:172.21.15.31:6789/0 conn(0x7fb74409aa00 legacy=0x7fb74409ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb742ffd640 1 -- 172.21.15.31:0/965012997 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440a2f10 con 0x7fb74409b520 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb742ffd640 1 --2- 172.21.15.31:0/965012997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409b520 0x7fb744138670 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ade86dee383f4b server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb7427fc640 1 -- 172.21.15.31:0/965012997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb738009ce0 con 0x7fb74409b520 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb7427fc640 1 -- 172.21.15.31:0/965012997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb738009e40 con 0x7fb74409b520 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb7427fc640 1 -- 172.21.15.31:0/965012997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb73800e360 con 0x7fb74409b520 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb749b5e640 1 -- 172.21.15.31:0/965012997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409b520 msgr2=0x7fb744138670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.426+0000 7fb749b5e640 1 --2- 172.21.15.31:0/965012997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409b520 0x7fb744138670 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.427+0000 7fb749b5e640 1 -- 172.21.15.31:0/965012997 shutdown_connections 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.427+0000 7fb749b5e640 1 --2- 172.21.15.31:0/965012997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409b520 0x7fb744138670 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.427+0000 7fb749b5e640 1 -- 172.21.15.31:0/965012997 >> 172.21.15.31:0/965012997 conn(0x7fb7440a3500 msgr2=0x7fb7440a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.427+0000 7fb749b5e640 1 -- 172.21.15.31:0/965012997 shutdown_connections 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.427+0000 7fb749b5e640 1 -- 172.21.15.31:0/965012997 wait complete. 2024-01-28T12:56:30.426 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.427+0000 7fb749b5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.428+0000 7fb749b5e640 1 Processor -- start 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.428+0000 7fb749b5e640 1 -- start start 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.428+0000 7fb749b5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409aa00 0x7fb74411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.428+0000 7fb749b5e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74411ed50 con 0x7fb74409aa00 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb7437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409aa00 0x7fb74411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb7437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409aa00 0x7fb74411ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56872/0 (socket says 172.21.15.31:56872) 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb7437fe640 1 -- 172.21.15.31:0/20119763 learned_addr learned my addr 172.21.15.31:0/20119763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb7437fe640 1 -- 172.21.15.31:0/20119763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744120450 con 0x7fb74409aa00 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb7437fe640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409aa00 0x7fb74411ca20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb72bfff640 1 -- 172.21.15.31:0/20119763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb7340059d0 con 0x7fb74409aa00 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb72bfff640 1 -- 172.21.15.31:0/20119763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb734005b30 con 0x7fb74409aa00 2024-01-28T12:56:30.429 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb72bfff640 1 -- 172.21.15.31:0/20119763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb734005dd0 con 0x7fb74409aa00 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb744120960 con 0x7fb74409aa00 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.429+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74411fef0 con 0x7fb74409aa00 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.430+0000 7fb72bfff640 1 -- 172.21.15.31:0/20119763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 44861+0+0 (crc 0 0 0) 0x7fb7340070b0 con 0x7fb74409aa00 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.430+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74411f100 con 0x7fb74409aa00 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.430+0000 7fb72bfff640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fb7300379b0 0x7fb730039ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.430+0000 7fb72bfff640 1 -- 172.21.15.31:0/20119763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7fb734046c40 con 0x7fb74409aa00 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.430+0000 7fb742ffd640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fb7300379b0 0x7fb730039ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:30.430 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.431+0000 7fb742ffd640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fb7300379b0 0x7fb730039ec0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:30.433 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.435+0000 7fb72bfff640 1 -- 172.21.15.31:0/20119763 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7fb73400d510 con 0x7fb74409aa00 2024-01-28T12:56:30.616 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.618+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb744120200 con 0x7fb74409aa00 2024-01-28T12:56:30.617 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.619+0000 7fb72bfff640 1 -- 172.21.15.31:0/20119763 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6898 (crc 0 0 0) 0x7fb73400d150 con 0x7fb74409aa00 2024-01-28T12:56:30.617 INFO:teuthology.misc.health.smithi031.stdout: 2024-01-28T12:56:30.617 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":11,"fsid":"68a362e5-2887-44cb-94c6-4a7b059316bf","created":"2024-01-28T12:56:20.671303+0000","modified":"2024-01-28T12:56:29.899791+0000","last_up_change":"2024-01-28T12:56:26.892641+0000","last_in_change":"2024-01-28T12:56:22.763590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-28T12:56:28.888517+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f07c134b-39dd-4d30-b1af-9fe992db79d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":507996357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6801","nonce":507996357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":507996357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6805","nonce":507996357}]},"public_addr":"172.21.15.31:6800/507996357","cluster_addr":"172.21.15.31:6801/507996357","heartbeat_back_addr":"172.21.15.31:6804/507996357","heartbeat_front_addr":"172.21.15.31:6805/507996357","state":["exists","up"]},{"osd":1,"uuid":"4d58f6cd-02f5-44e3-9b51-9e99f3720656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":2734761187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2734761187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2734761187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":2734761187}]},"public_addr":"172.21.15.31:6803/2734761187","cluster_addr":"172.21.15.31:6802/2734761187","heartbeat_back_addr":"172.21.15.31:6806/2734761187","heartbeat_front_addr":"172.21.15.31:6807/2734761187","state":["exists","up"]},{"osd":2,"uuid":"05ce0150-eebc-405f-8fb8-fe1ef00b7cd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":964334066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":964334066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":964334066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":964334066}]},"public_addr":"172.21.15.137:6804/964334066","cluster_addr":"172.21.15.137:6805/964334066","heartbeat_back_addr":"172.21.15.137:6809/964334066","heartbeat_front_addr":"172.21.15.137:6808/964334066","state":["exists","up"]},{"osd":3,"uuid":"bd000973-288c-4ebb-a3f5-871638c8e946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3189673583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":3189673583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":3189673583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3189673583}]},"public_addr":"172.21.15.137:6802/3189673583","cluster_addr":"172.21.15.137:6803/3189673583","heartbeat_back_addr":"172.21.15.137:6807/3189673583","heartbeat_front_addr":"172.21.15.137:6806/3189673583","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-28T12:56:30.619 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fb7300379b0 msgr2=0x7fb730039ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fb7300379b0 0x7fb730039ec0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409aa00 msgr2=0x7fb74411ca20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409aa00 0x7fb74411ca20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 shutdown_connections 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fb7300379b0 0x7fb730039ec0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 --2- 172.21.15.31:0/20119763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74409aa00 0x7fb74411ca20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 >> 172.21.15.31:0/20119763 conn(0x7fb7440a3500 msgr2=0x7fb7441bb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 shutdown_connections 2024-01-28T12:56:30.620 INFO:teuthology.misc.health.smithi031.stderr:2024-01-28T12:56:30.621+0000 7fb749b5e640 1 -- 172.21.15.31:0/20119763 wait complete. 2024-01-28T12:56:30.629 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-01-28T12:56:30.629 INFO:tasks.ceph:Creating RBD pool 2024-01-28T12:56:30.629 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2024-01-28T12:56:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.733+0000 7fdee4b1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.740+0000 7fdee4b1e640 1 Processor -- start 2024-01-28T12:56:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.740+0000 7fdee4b1e640 1 -- start start 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdee4b1e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdee01300e0 0x7fdee01304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdee4b1e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee009e400 con 0x7fdee01330d0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdee4b1e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee009ce70 con 0x7fdee01300e0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdede59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdee01300e0 0x7fdee01304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdeddd9b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdee01330d0 0x7fdee01334e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60838/0 (socket says 172.21.15.31:60838) 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdeddd9b640 1 -- 172.21.15.31:0/741814924 learned_addr learned my addr 172.21.15.31:0/741814924 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdedd59a640 1 -- 172.21.15.31:0/741814924 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872627370 0 0) 0x7fdee009e400 con 0x7fdee01330d0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdedd59a640 1 -- 172.21.15.31:0/741814924 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec80034f0 con 0x7fdee01330d0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdede59c640 1 -- 172.21.15.31:0/741814924 >> v1:172.21.15.31:6789/0 conn(0x7fdee01330d0 legacy=0x7fdee01334e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdede59c640 1 -- 172.21.15.31:0/741814924 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee009cb60 con 0x7fdee01300e0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdede59c640 1 --2- 172.21.15.31:0/741814924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee01304f0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e4e12fe16904e server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdedd59a640 1 -- 172.21.15.31:0/741814924 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fded0009ce0 con 0x7fdee01300e0 2024-01-28T12:56:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.741+0000 7fdedd59a640 1 -- 172.21.15.31:0/741814924 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fded0009e40 con 0x7fdee01300e0 2024-01-28T12:56:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdedd59a640 1 -- 172.21.15.31:0/741814924 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fded000e360 con 0x7fdee01300e0 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdee4b1e640 1 -- 172.21.15.31:0/741814924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 msgr2=0x7fdee01304f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdee4b1e640 1 --2- 172.21.15.31:0/741814924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee01304f0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdee4b1e640 1 -- 172.21.15.31:0/741814924 shutdown_connections 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdee4b1e640 1 --2- 172.21.15.31:0/741814924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee01304f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdee4b1e640 1 -- 172.21.15.31:0/741814924 >> 172.21.15.31:0/741814924 conn(0x7fdee00a6000 msgr2=0x7fdee00989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdee4b1e640 1 -- 172.21.15.31:0/741814924 shutdown_connections 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.742+0000 7fdee4b1e640 1 -- 172.21.15.31:0/741814924 wait complete. 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.743+0000 7fdee4b1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.744+0000 7fdee4b1e640 1 Processor -- start 2024-01-28T12:56:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.744+0000 7fdee4b1e640 1 -- start start 2024-01-28T12:56:30.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.744+0000 7fdee4b1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee011bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:30.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.744+0000 7fdee4b1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee009c2c0 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdede59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee011bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdede59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee011bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56898/0 (socket says 172.21.15.31:56898) 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdede59c640 1 -- 172.21.15.31:0/267301924 learned_addr learned my addr 172.21.15.31:0/267301924 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdede59c640 1 -- 172.21.15.31:0/267301924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee006afb0 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdede59c640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee011bbe0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fded0014410 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fded0014570 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fded0014810 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee011edd0 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.745+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee011fb40 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.746+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 44861+0+0 (crc 0 0 0) 0x7fded0014a10 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.746+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee011cfd0 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.746+0000 7fdeceffd640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fdec0037960 0x7fdec0039e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.746+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7fded00481d0 con 0x7fdee01300e0 2024-01-28T12:56:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.746+0000 7fdeddd9b640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fdec0037960 0x7fdec0039e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:30.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.747+0000 7fdeddd9b640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fdec0037960 0x7fdec0039e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:30.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.751+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7fdee011cfd0 con 0x7fdee01300e0 2024-01-28T12:56:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:30.929+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fdee006afb0 con 0x7fdee01300e0 2024-01-28T12:56:31.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.687+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7fded0013920 con 0x7fdee01300e0 2024-01-28T12:56:31.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.906+0000 7fdeceffd640 1 -- 172.21.15.31:0/267301924 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (crc 0 0 0) 0x7fded0005960 con 0x7fdee01300e0 2024-01-28T12:56:31.904 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2024-01-28T12:56:31.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fdec0037960 msgr2=0x7fdec0039e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:31.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fdec0037960 0x7fdec0039e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:31.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 msgr2=0x7fdee011bbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:31.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee011bbe0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:31.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 shutdown_connections 2024-01-28T12:56:31.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fdec0037960 0x7fdec0039e70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:31.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 --2- 172.21.15.31:0/267301924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdee01300e0 0x7fdee011bbe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:31.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 >> 172.21.15.31:0/267301924 conn(0x7fdee00a6000 msgr2=0x7fdee01b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:31.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 shutdown_connections 2024-01-28T12:56:31.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:31.909+0000 7fdee4b1e640 1 -- 172.21.15.31:0/267301924 wait complete. 2024-01-28T12:56:31.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-01-28T12:56:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.033+0000 7f513846d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.042+0000 7f513846d640 1 Processor -- start 2024-01-28T12:56:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.042+0000 7f513846d640 1 -- start start 2024-01-28T12:56:32.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f513846d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f513012faf0 0x7f513012ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:32.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f513846d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513012f6c0 con 0x7f5130098e30 2024-01-28T12:56:32.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f513846d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300981f0 con 0x7f513012faf0 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5136209640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f513012faf0 0x7f513012ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5135a08640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5130098e30 0x7f5130099240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60848/0 (socket says 172.21.15.31:60848) 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5135a08640 1 -- 172.21.15.31:0/3124492219 learned_addr learned my addr 172.21.15.31:0/3124492219 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5136209640 1 -- 172.21.15.31:0/3124492219 >> v1:172.21.15.31:6789/0 conn(0x7f5130098e30 legacy=0x7f5130099240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5136209640 1 -- 172.21.15.31:0/3124492219 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513009ae40 con 0x7f513012faf0 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5135207640 1 -- 172.21.15.31:0/3124492219 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870129578 0 0) 0x7f513012f6c0 con 0x7f5130098e30 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5136209640 1 --2- 172.21.15.31:0/3124492219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513012faf0 0x7f513012ff00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce71e3a1f4a613 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.043+0000 7f5135207640 1 -- 172.21.15.31:0/3124492219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f512c009ce0 con 0x7f513012faf0 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f5135207640 1 -- 172.21.15.31:0/3124492219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f512c009e40 con 0x7f513012faf0 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f5135207640 1 -- 172.21.15.31:0/3124492219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f512c00e360 con 0x7f513012faf0 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 1 -- 172.21.15.31:0/3124492219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513012faf0 msgr2=0x7f513012ff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 1 --2- 172.21.15.31:0/3124492219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513012faf0 0x7f513012ff00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 1 -- 172.21.15.31:0/3124492219 shutdown_connections 2024-01-28T12:56:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 1 --2- 172.21.15.31:0/3124492219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513012faf0 0x7f513012ff00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 1 -- 172.21.15.31:0/3124492219 >> 172.21.15.31:0/3124492219 conn(0x7f513009efa0 msgr2=0x7f513009f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 1 -- 172.21.15.31:0/3124492219 shutdown_connections 2024-01-28T12:56:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 1 -- 172.21.15.31:0/3124492219 wait complete. 2024-01-28T12:56:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.044+0000 7f513846d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.045+0000 7f513846d640 1 Processor -- start 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.045+0000 7f513846d640 1 -- start start 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.046+0000 7f513846d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5130098e30 0x7f513012b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.046+0000 7f513846d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513008a590 con 0x7f5130098e30 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.046+0000 7f5136209640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5130098e30 0x7f513012b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.046+0000 7f5136209640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5130098e30 0x7f513012b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56924/0 (socket says 172.21.15.31:56924) 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.046+0000 7f5136209640 1 -- 172.21.15.31:0/2791164793 learned_addr learned my addr 172.21.15.31:0/2791164793 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.046+0000 7f5136209640 1 -- 172.21.15.31:0/2791164793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513006be70 con 0x7f5130098e30 2024-01-28T12:56:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.047+0000 7f5136209640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5130098e30 0x7f513012b210 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.047+0000 7f5126ffd640 1 -- 172.21.15.31:0/2791164793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f512c018040 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.047+0000 7f5126ffd640 1 -- 172.21.15.31:0/2791164793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f512c00e640 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.047+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513008acd0 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.047+0000 7f5126ffd640 1 -- 172.21.15.31:0/2791164793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f512c00e8e0 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.047+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513012af00 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.048+0000 7f5126ffd640 1 -- 172.21.15.31:0/2791164793 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f512c01b040 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.048+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130075440 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.048+0000 7f5126ffd640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5114037a40 0x7f5114039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.048+0000 7f5126ffd640 1 -- 172.21.15.31:0/2791164793 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2661+0+0 (crc 0 0 0) 0x7f512c04a040 con 0x7f5130098e30 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.048+0000 7f5135a08640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5114037a40 0x7f5114039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.049+0000 7f5135a08640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5114037a40 0x7f5114039f50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.053+0000 7f5126ffd640 1 -- 172.21.15.31:0/2791164793 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f512c018380 con 0x7f5130098e30 2024-01-28T12:56:32.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.231+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f513008b5a0 con 0x7f5130098e30 2024-01-28T12:56:32.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.923+0000 7f5126ffd640 1 -- 172.21.15.31:0/2791164793 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (crc 0 0 0) 0x7f512c0181a0 con 0x7f5130098e30 2024-01-28T12:56:32.921 INFO:teuthology.orchestra.run.smithi031.stderr:enabled application 'rbd' on pool 'rbd' 2024-01-28T12:56:32.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5114037a40 msgr2=0x7f5114039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:32.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5114037a40 0x7f5114039f50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5130098e30 msgr2=0x7f513012b210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5130098e30 0x7f513012b210 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 shutdown_connections 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5114037a40 0x7f5114039f50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 --2- 172.21.15.31:0/2791164793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5130098e30 0x7f513012b210 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 >> 172.21.15.31:0/2791164793 conn(0x7f513009efa0 msgr2=0x7f51300985a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 shutdown_connections 2024-01-28T12:56:32.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:32.925+0000 7f513846d640 1 -- 172.21.15.31:0/2791164793 wait complete. 2024-01-28T12:56:32.938 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-28T12:56:32.938 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-28T12:56:32.939 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-28T12:56:32.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-28T12:56:32.939 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-28T12:56:33.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.055+0000 7fbe3899e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:33.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.059+0000 7fbe3899e640 1 Processor -- start 2024-01-28T12:56:33.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.059+0000 7fbe3899e640 1 -- start start 2024-01-28T12:56:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.059+0000 7fbe3899e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbe341343c0 0x7fbe341367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.059+0000 7fbe3899e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe34132000 con 0x7fbe341314e0 2024-01-28T12:56:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.059+0000 7fbe3899e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3406f720 con 0x7fbe341343c0 2024-01-28T12:56:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe31d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbe341343c0 0x7fbe341367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe3259c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbe341314e0 0x7fbe341318f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60854/0 (socket says 172.21.15.31:60854) 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe3259c640 1 -- 172.21.15.31:0/2610408109 learned_addr learned my addr 172.21.15.31:0/2610408109 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe31d9b640 1 -- 172.21.15.31:0/2610408109 >> v1:172.21.15.31:6789/0 conn(0x7fbe341314e0 legacy=0x7fbe341318f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe31d9b640 1 -- 172.21.15.31:0/2610408109 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3406e9c0 con 0x7fbe341343c0 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe3159a640 1 -- 172.21.15.31:0/2610408109 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231446551 0 0) 0x7fbe24001110 con 0x7fbe341314e0 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe31d9b640 1 --2- 172.21.15.31:0/2610408109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341343c0 0x7fbe341367e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ea479c965f2a8a server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe3159a640 1 -- 172.21.15.31:0/2610408109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbe28009ce0 con 0x7fbe341343c0 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe3159a640 1 -- 172.21.15.31:0/2610408109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbe28009e40 con 0x7fbe341343c0 2024-01-28T12:56:33.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.060+0000 7fbe3159a640 1 -- 172.21.15.31:0/2610408109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbe2800e360 con 0x7fbe341343c0 2024-01-28T12:56:33.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 1 -- 172.21.15.31:0/2610408109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341343c0 msgr2=0x7fbe341367e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 1 --2- 172.21.15.31:0/2610408109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341343c0 0x7fbe341367e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 1 -- 172.21.15.31:0/2610408109 shutdown_connections 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 1 --2- 172.21.15.31:0/2610408109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341343c0 0x7fbe341367e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 1 -- 172.21.15.31:0/2610408109 >> 172.21.15.31:0/2610408109 conn(0x7fbe3409e4b0 msgr2=0x7fbe3409e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 1 -- 172.21.15.31:0/2610408109 shutdown_connections 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 1 -- 172.21.15.31:0/2610408109 wait complete. 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.061+0000 7fbe3899e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.062+0000 7fbe3899e640 1 Processor -- start 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.062+0000 7fbe3899e640 1 -- start start 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.063+0000 7fbe3899e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341314e0 0x7fbe3411dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.063+0000 7fbe3899e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34120130 con 0x7fbe341314e0 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.063+0000 7fbe3259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341314e0 0x7fbe3411dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.063+0000 7fbe3259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341314e0 0x7fbe3411dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56936/0 (socket says 172.21.15.31:56936) 2024-01-28T12:56:33.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.063+0000 7fbe3259c640 1 -- 172.21.15.31:0/2087859110 learned_addr learned my addr 172.21.15.31:0/2087859110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:33.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.063+0000 7fbe3259c640 1 -- 172.21.15.31:0/2087859110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3411f850 con 0x7fbe341314e0 2024-01-28T12:56:33.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.063+0000 7fbe3259c640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341314e0 0x7fbe3411dc30 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.064+0000 7fbe22ffd640 1 -- 172.21.15.31:0/2087859110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbe24004af0 con 0x7fbe341314e0 2024-01-28T12:56:33.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.064+0000 7fbe22ffd640 1 -- 172.21.15.31:0/2087859110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbe24004c50 con 0x7fbe341314e0 2024-01-28T12:56:33.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.064+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3411eff0 con 0x7fbe341314e0 2024-01-28T12:56:33.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.064+0000 7fbe22ffd640 1 -- 172.21.15.31:0/2087859110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbe24007120 con 0x7fbe341314e0 2024-01-28T12:56:33.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.064+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe341207b0 con 0x7fbe341314e0 2024-01-28T12:56:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.064+0000 7fbe22ffd640 1 -- 172.21.15.31:0/2087859110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7fbe24007320 con 0x7fbe341314e0 2024-01-28T12:56:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.065+0000 7fbe22ffd640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fbe140379f0 0x7fbe14039f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.065+0000 7fbe22ffd640 1 -- 172.21.15.31:0/2087859110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fbe24046ed0 con 0x7fbe341314e0 2024-01-28T12:56:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.065+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe34132000 con 0x7fbe341314e0 2024-01-28T12:56:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.065+0000 7fbe31d9b640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fbe140379f0 0x7fbe14039f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.065+0000 7fbe31d9b640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fbe140379f0 0x7fbe14039f00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.069+0000 7fbe22ffd640 1 -- 172.21.15.31:0/2087859110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7fbe34132000 con 0x7fbe341314e0 2024-01-28T12:56:33.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.298+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbe34132000 con 0x7fbe341314e0 2024-01-28T12:56:33.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.300+0000 7fbe22ffd640 1 -- 172.21.15.31:0/2087859110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+86127 (crc 0 0 0) 0x7fbe34132000 con 0x7fbe341314e0 2024-01-28T12:56:33.299 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.302+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fbe140379f0 msgr2=0x7fbe14039f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.302+0000 7fbe3899e640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fbe140379f0 0x7fbe14039f00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.302+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341314e0 msgr2=0x7fbe3411dc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.302+0000 7fbe3899e640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341314e0 0x7fbe3411dc30 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.303+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 shutdown_connections 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.303+0000 7fbe3899e640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fbe140379f0 0x7fbe14039f00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.303+0000 7fbe3899e640 1 --2- 172.21.15.31:0/2087859110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe341314e0 0x7fbe3411dc30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.303+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 >> 172.21.15.31:0/2087859110 conn(0x7fbe3409e4b0 msgr2=0x7fbe340a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.303+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 shutdown_connections 2024-01-28T12:56:33.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.303+0000 7fbe3899e640 1 -- 172.21.15.31:0/2087859110 wait complete. 2024-01-28T12:56:33.311 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1465424679},{"type":"v1","addr":"172.21.15.137:6801","nonce":1465424679}]},"active_addr":"172.21.15.137:6801/1465424679","active_change":"2024-01-28T12:56:24.840548+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":2796857248}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":4145485134}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":1270122317}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":4236282818}]}]} 2024-01-28T12:56:33.313 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-28T12:56:33.313 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-28T12:56:33.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-28T12:56:33.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.417+0000 7f897e46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:33.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.421+0000 7f897e46d640 1 Processor -- start 2024-01-28T12:56:33.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.421+0000 7f897e46d640 1 -- start start 2024-01-28T12:56:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f897e46d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f897809ab60 0x7f8978135540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f897e46d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897806e800 con 0x7f897809a040 2024-01-28T12:56:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f897e46d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780986a0 con 0x7f897809ab60 2024-01-28T12:56:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f89777fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f897809ab60 0x7f8978135540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f8977fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f897809a040 0x7f897809a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60870/0 (socket says 172.21.15.31:60870) 2024-01-28T12:56:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f8977fff640 1 -- 172.21.15.31:0/1190221228 learned_addr learned my addr 172.21.15.31:0/1190221228 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f89777fe640 1 -- 172.21.15.31:0/1190221228 >> v1:172.21.15.31:6789/0 conn(0x7f897809a040 legacy=0x7f897809a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f89777fe640 1 -- 172.21.15.31:0/1190221228 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897806ef20 con 0x7f897809ab60 2024-01-28T12:56:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f89777fe640 1 --2- 172.21.15.31:0/1190221228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809ab60 0x7f8978135540 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0afedaaf97cab64 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f8976ffd640 1 -- 172.21.15.31:0/1190221228 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f896c009ce0 con 0x7f897809ab60 2024-01-28T12:56:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f8976ffd640 1 -- 172.21.15.31:0/1190221228 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f896c009e40 con 0x7f897809ab60 2024-01-28T12:56:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.422+0000 7f8976ffd640 1 -- 172.21.15.31:0/1190221228 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f896c00e360 con 0x7f897809ab60 2024-01-28T12:56:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 1 -- 172.21.15.31:0/1190221228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809ab60 msgr2=0x7f8978135540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 1 --2- 172.21.15.31:0/1190221228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809ab60 0x7f8978135540 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 1 -- 172.21.15.31:0/1190221228 shutdown_connections 2024-01-28T12:56:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 1 --2- 172.21.15.31:0/1190221228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809ab60 0x7f8978135540 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 1 -- 172.21.15.31:0/1190221228 >> 172.21.15.31:0/1190221228 conn(0x7f897809e620 msgr2=0x7f897809bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 1 -- 172.21.15.31:0/1190221228 shutdown_connections 2024-01-28T12:56:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 1 -- 172.21.15.31:0/1190221228 wait complete. 2024-01-28T12:56:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.423+0000 7f897e46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.424+0000 7f897e46d640 1 Processor -- start 2024-01-28T12:56:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.424+0000 7f897e46d640 1 -- start start 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.425+0000 7f897e46d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809a040 0x7f89781b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.425+0000 7f897e46d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89781b4a50 con 0x7f897809a040 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.425+0000 7f8977fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809a040 0x7f89781b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.425+0000 7f8977fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809a040 0x7f89781b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56944/0 (socket says 172.21.15.31:56944) 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.425+0000 7f8977fff640 1 -- 172.21.15.31:0/3189754357 learned_addr learned my addr 172.21.15.31:0/3189754357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.425+0000 7f8977fff640 1 -- 172.21.15.31:0/3189754357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89781b5e40 con 0x7f897809a040 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.426+0000 7f8977fff640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809a040 0x7f89781b60e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.426+0000 7f8974ff9640 1 -- 172.21.15.31:0/3189754357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f89680058a0 con 0x7f897809a040 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.426+0000 7f8974ff9640 1 -- 172.21.15.31:0/3189754357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8968005a00 con 0x7f897809a040 2024-01-28T12:56:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.426+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89781b7e10 con 0x7f897809a040 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.426+0000 7f8974ff9640 1 -- 172.21.15.31:0/3189754357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8968005ca0 con 0x7f897809a040 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.426+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897806bf40 con 0x7f897809a040 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.427+0000 7f8974ff9640 1 -- 172.21.15.31:0/3189754357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f89680070b0 con 0x7f897809a040 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.427+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8978075440 con 0x7f897809a040 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.427+0000 7f8974ff9640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f8964037a40 0x7f8964039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.427+0000 7f8974ff9640 1 -- 172.21.15.31:0/3189754357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8968046df0 con 0x7f897809a040 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.427+0000 7f89777fe640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f8964037a40 0x7f8964039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.428+0000 7f89777fe640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f8964037a40 0x7f8964039f50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.431+0000 7f8974ff9640 1 -- 172.21.15.31:0/3189754357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f896800d9d0 con 0x7f897809a040 2024-01-28T12:56:33.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.614+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f897806b740 con 0x7f897809a040 2024-01-28T12:56:33.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.615+0000 7f8974ff9640 1 -- 172.21.15.31:0/3189754357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8657 (crc 0 0 0) 0x7f896800d5c0 con 0x7f897809a040 2024-01-28T12:56:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:56:33.613 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":14,"fsid":"68a362e5-2887-44cb-94c6-4a7b059316bf","created":"2024-01-28T12:56:20.671303+0000","modified":"2024-01-28T12:56:32.916712+0000","last_up_change":"2024-01-28T12:56:26.892641+0000","last_in_change":"2024-01-28T12:56:22.763590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-28T12:56:28.888517+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-28T12:56:30.948909+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f07c134b-39dd-4d30-b1af-9fe992db79d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":507996357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6801","nonce":507996357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":507996357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6805","nonce":507996357}]},"public_addr":"172.21.15.31:6800/507996357","cluster_addr":"172.21.15.31:6801/507996357","heartbeat_back_addr":"172.21.15.31:6804/507996357","heartbeat_front_addr":"172.21.15.31:6805/507996357","state":["exists","up"]},{"osd":1,"uuid":"4d58f6cd-02f5-44e3-9b51-9e99f3720656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":2734761187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2734761187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2734761187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":2734761187}]},"public_addr":"172.21.15.31:6803/2734761187","cluster_addr":"172.21.15.31:6802/2734761187","heartbeat_back_addr":"172.21.15.31:6806/2734761187","heartbeat_front_addr":"172.21.15.31:6807/2734761187","state":["exists","up"]},{"osd":2,"uuid":"05ce0150-eebc-405f-8fb8-fe1ef00b7cd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":964334066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":964334066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":964334066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":964334066}]},"public_addr":"172.21.15.137:6804/964334066","cluster_addr":"172.21.15.137:6805/964334066","heartbeat_back_addr":"172.21.15.137:6809/964334066","heartbeat_front_addr":"172.21.15.137:6808/964334066","state":["exists","up"]},{"osd":3,"uuid":"bd000973-288c-4ebb-a3f5-871638c8e946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3189673583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":3189673583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":3189673583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3189673583}]},"public_addr":"172.21.15.137:6802/3189673583","cluster_addr":"172.21.15.137:6803/3189673583","heartbeat_back_addr":"172.21.15.137:6807/3189673583","heartbeat_front_addr":"172.21.15.137:6806/3189673583","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-28T12:56:33.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f8964037a40 msgr2=0x7f8964039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f8964037a40 0x7f8964039f50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809a040 msgr2=0x7f89781b60e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809a040 0x7f89781b60e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 shutdown_connections 2024-01-28T12:56:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f8964037a40 0x7f8964039f50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 --2- 172.21.15.31:0/3189754357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897809a040 0x7f89781b60e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 >> 172.21.15.31:0/3189754357 conn(0x7f897809e620 msgr2=0x7f89780a0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 shutdown_connections 2024-01-28T12:56:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.617+0000 7f897e46d640 1 -- 172.21.15.31:0/3189754357 wait complete. 2024-01-28T12:56:33.625 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-28T12:56:33.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-28T12:56:33.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.729+0000 7f43c1f5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:33.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.733+0000 7f43c1f5e640 1 Processor -- start 2024-01-28T12:56:33.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.733+0000 7f43c1f5e640 1 -- start start 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.734+0000 7f43c1f5e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43bc09adb0 0x7f43bc09b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.734+0000 7f43c1f5e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc09a420 con 0x7f43bc133220 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.734+0000 7f43c1f5e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc09e4d0 con 0x7f43bc09adb0 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.734+0000 7f43bb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43bc09adb0 0x7f43bc09b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.734+0000 7f43baffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f43bc133220 0x7f43bc133630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60880/0 (socket says 172.21.15.31:60880) 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.734+0000 7f43baffd640 1 -- 172.21.15.31:0/2446267023 learned_addr learned my addr 172.21.15.31:0/2446267023 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2446267023 >> v1:172.21.15.31:6789/0 conn(0x7f43bc133220 legacy=0x7f43bc133630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43bb7fe640 1 -- 172.21.15.31:0/2446267023 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc09a960 con 0x7f43bc09adb0 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43bb7fe640 1 --2- 172.21.15.31:0/2446267023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc09b1c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459a1a4afa0148ac server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43ba7fc640 1 -- 172.21.15.31:0/2446267023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43ac009ce0 con 0x7f43bc09adb0 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43ba7fc640 1 -- 172.21.15.31:0/2446267023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f43ac009e40 con 0x7f43bc09adb0 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43ba7fc640 1 -- 172.21.15.31:0/2446267023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43ac00e360 con 0x7f43bc09adb0 2024-01-28T12:56:33.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43c1f5e640 1 -- 172.21.15.31:0/2446267023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 msgr2=0x7f43bc09b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43c1f5e640 1 --2- 172.21.15.31:0/2446267023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc09b1c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43c1f5e640 1 -- 172.21.15.31:0/2446267023 shutdown_connections 2024-01-28T12:56:33.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43c1f5e640 1 --2- 172.21.15.31:0/2446267023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc09b1c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.735+0000 7f43c1f5e640 1 -- 172.21.15.31:0/2446267023 >> 172.21.15.31:0/2446267023 conn(0x7f43bc0a6060 msgr2=0x7f43bc0989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:33.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.736+0000 7f43c1f5e640 1 -- 172.21.15.31:0/2446267023 shutdown_connections 2024-01-28T12:56:33.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.736+0000 7f43c1f5e640 1 -- 172.21.15.31:0/2446267023 wait complete. 2024-01-28T12:56:33.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.736+0000 7f43c1f5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.737+0000 7f43c1f5e640 1 Processor -- start 2024-01-28T12:56:33.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.737+0000 7f43c1f5e640 1 -- start start 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.737+0000 7f43c1f5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc11c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.737+0000 7f43c1f5e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc130950 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc11c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc11c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56970/0 (socket says 172.21.15.31:56970) 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43bb7fe640 1 -- 172.21.15.31:0/3388276335 learned_addr learned my addr 172.21.15.31:0/3388276335 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43bb7fe640 1 -- 172.21.15.31:0/3388276335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc11f180 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43bb7fe640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc11c9f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43a3fff640 1 -- 172.21.15.31:0/3388276335 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43ac014540 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43a3fff640 1 -- 172.21.15.31:0/3388276335 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f43ac0146a0 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc130b80 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43a3fff640 1 -- 172.21.15.31:0/3388276335 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43ac014940 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.738+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc1209b0 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.739+0000 7f43a3fff640 1 -- 172.21.15.31:0/3388276335 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f43ac014af0 con 0x7f43bc09adb0 2024-01-28T12:56:33.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.739+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc075440 con 0x7f43bc09adb0 2024-01-28T12:56:33.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.739+0000 7f43a3fff640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f43a8037a40 0x7f43a8039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:33.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.739+0000 7f43a3fff640 1 -- 172.21.15.31:0/3388276335 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f43ac019040 con 0x7f43bc09adb0 2024-01-28T12:56:33.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.740+0000 7f43baffd640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f43a8037a40 0x7f43a8039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:33.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.740+0000 7f43baffd640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f43a8037a40 0x7f43a8039f50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:33.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.744+0000 7f43a3fff640 1 -- 172.21.15.31:0/3388276335 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f43ac018190 con 0x7f43bc09adb0 2024-01-28T12:56:33.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.923+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43bc0742a0 con 0x7f43bc09adb0 2024-01-28T12:56:33.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.924+0000 7f43a3fff640 1 -- 172.21.15.31:0/3388276335 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8657 (crc 0 0 0) 0x7f43ac00fa90 con 0x7f43bc09adb0 2024-01-28T12:56:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:56:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":14,"fsid":"68a362e5-2887-44cb-94c6-4a7b059316bf","created":"2024-01-28T12:56:20.671303+0000","modified":"2024-01-28T12:56:32.916712+0000","last_up_change":"2024-01-28T12:56:26.892641+0000","last_in_change":"2024-01-28T12:56:22.763590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-28T12:56:28.888517+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-01-28T12:56:30.948909+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f07c134b-39dd-4d30-b1af-9fe992db79d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":507996357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6801","nonce":507996357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":507996357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6805","nonce":507996357}]},"public_addr":"172.21.15.31:6800/507996357","cluster_addr":"172.21.15.31:6801/507996357","heartbeat_back_addr":"172.21.15.31:6804/507996357","heartbeat_front_addr":"172.21.15.31:6805/507996357","state":["exists","up"]},{"osd":1,"uuid":"4d58f6cd-02f5-44e3-9b51-9e99f3720656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":2734761187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2734761187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2734761187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":2734761187}]},"public_addr":"172.21.15.31:6803/2734761187","cluster_addr":"172.21.15.31:6802/2734761187","heartbeat_back_addr":"172.21.15.31:6806/2734761187","heartbeat_front_addr":"172.21.15.31:6807/2734761187","state":["exists","up"]},{"osd":2,"uuid":"05ce0150-eebc-405f-8fb8-fe1ef00b7cd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":964334066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":964334066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":964334066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":964334066}]},"public_addr":"172.21.15.137:6804/964334066","cluster_addr":"172.21.15.137:6805/964334066","heartbeat_back_addr":"172.21.15.137:6809/964334066","heartbeat_front_addr":"172.21.15.137:6808/964334066","state":["exists","up"]},{"osd":3,"uuid":"bd000973-288c-4ebb-a3f5-871638c8e946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3189673583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":3189673583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":3189673583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3189673583}]},"public_addr":"172.21.15.137:6802/3189673583","cluster_addr":"172.21.15.137:6803/3189673583","heartbeat_back_addr":"172.21.15.137:6807/3189673583","heartbeat_front_addr":"172.21.15.137:6806/3189673583","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-28T12:56:33.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f43a8037a40 msgr2=0x7f43a8039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f43a8037a40 0x7f43a8039f50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 msgr2=0x7f43bc11c9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:33.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc11c9f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 shutdown_connections 2024-01-28T12:56:33.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f43a8037a40 0x7f43a8039f50 crc :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 --2- 172.21.15.31:0/3388276335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc09adb0 0x7f43bc11c9f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:33.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 >> 172.21.15.31:0/3388276335 conn(0x7f43bc0a6060 msgr2=0x7f43bc1bb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:33.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 shutdown_connections 2024-01-28T12:56:33.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:33.926+0000 7f43c1f5e640 1 -- 172.21.15.31:0/3388276335 wait complete. 2024-01-28T12:56:33.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-28T12:56:33.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-28T12:56:33.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-28T12:56:33.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-28T12:56:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.050+0000 7f7bb062d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.050+0000 7f56b2e64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.051+0000 7fc1e026d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.050+0000 7f08c7060640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.055+0000 7f7bb062d640 1 Processor -- start 2024-01-28T12:56:34.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.055+0000 7fc1e026d640 1 Processor -- start 2024-01-28T12:56:34.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.055+0000 7f7bb062d640 1 -- start start 2024-01-28T12:56:34.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.055+0000 7fc1e026d640 1 -- start start 2024-01-28T12:56:34.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f56b2e64640 1 Processor -- start 2024-01-28T12:56:34.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f08c7060640 1 Processor -- start 2024-01-28T12:56:34.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f56b2e64640 1 -- start start 2024-01-28T12:56:34.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f08c7060640 1 -- start start 2024-01-28T12:56:34.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7fc1e026d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1d81363e0 0x7fc1d8138800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f56b2e64640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56ac099150 0x7f56ac136630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f56b2e64640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ac09b610 con 0x7f56ac098630 2024-01-28T12:56:34.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f56b2e64640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0700e0 con 0x7f56ac099150 2024-01-28T12:56:34.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7fc1e026d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d809e400 con 0x7fc1d81300e0 2024-01-28T12:56:34.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7fc1e026d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d809ce70 con 0x7fc1d81363e0 2024-01-28T12:56:34.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f7bb062d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ba809aa00 0x7f7ba809ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f7bb062d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba809e410 con 0x7f7ba809b350 2024-01-28T12:56:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.056+0000 7f7bb062d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba809cdd0 con 0x7f7ba809aa00 2024-01-28T12:56:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7fc1dd808640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1d81363e0 0x7fc1d8138800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08c7060640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08c0098d40 0x7f08c01304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7fc1dd808640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1d81363e0 0x7fc1d8138800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56982/0 (socket says 172.21.15.31:56982) 2024-01-28T12:56:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7fc1dd808640 1 -- 172.21.15.31:0/331663352 learned_addr learned my addr 172.21.15.31:0/331663352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f7bae3c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ba809aa00 0x7f7ba809ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f7bae3c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ba809aa00 0x7f7ba809ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56990/0 (socket says 172.21.15.31:56990) 2024-01-28T12:56:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f7bae3c9640 1 -- 172.21.15.31:0/4011401716 learned_addr learned my addr 172.21.15.31:0/4011401716 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08c7060640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c009a3a0 con 0x7f08c006d260 2024-01-28T12:56:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08c7060640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c006dc30 con 0x7f08c0098d40 2024-01-28T12:56:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f56abfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56ac099150 0x7f56ac136630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f56abfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56ac099150 0x7f56ac136630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56974/0 (socket says 172.21.15.31:56974) 2024-01-28T12:56:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f56abfff640 1 -- 172.21.15.31:0/177110586 learned_addr learned my addr 172.21.15.31:0/177110586 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08c4dfc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f08c006d260 0x7f08c0098630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60926/0 (socket says 172.21.15.31:60926) 2024-01-28T12:56:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08c4dfc640 1 -- 172.21.15.31:0/4148111608 learned_addr learned my addr 172.21.15.31:0/4148111608 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7fc1dd808640 1 -- 172.21.15.31:0/331663352 >> v1:172.21.15.31:6789/0 conn(0x7fc1d81300e0 legacy=0x7fc1d81304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-28T12:56:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7fc1dd808640 1 -- 172.21.15.31:0/331663352 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d809cb60 con 0x7fc1d81363e0 2024-01-28T12:56:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08bffff640 1 --2- 172.21.15.31:0/4148111608 >> v2:172.21.15.31:3300/0 conn(0x7f08c0098d40 0x7f08c01304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f56abfff640 1 -- 172.21.15.31:0/177110586 >> v1:172.21.15.31:6789/0 conn(0x7f56ac098630 legacy=0x7f56ac098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-28T12:56:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f56abfff640 1 -- 172.21.15.31:0/177110586 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac09b390 con 0x7f56ac099150 2024-01-28T12:56:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f56abfff640 1 --2- 172.21.15.31:0/177110586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac099150 0x7f56ac136630 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d7f0df46b0c5cc server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7fc1dd808640 1 --2- 172.21.15.31:0/331663352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81363e0 0x7fc1d8138800 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd16f2f3671dd9b server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08bf7fe640 1 -- 172.21.15.31:0/4148111608 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065982031 0 0) 0x7f08c009a3a0 con 0x7f08c006d260 2024-01-28T12:56:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f08bf7fe640 1 -- 172.21.15.31:0/4148111608 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08b00034f0 con 0x7f08c006d260 2024-01-28T12:56:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f7bae3c9640 1 -- 172.21.15.31:0/4011401716 >> v1:172.21.15.31:6789/0 conn(0x7f7ba809b350 legacy=0x7f7ba812fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.057+0000 7f7bae3c9640 1 -- 172.21.15.31:0/4011401716 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba809d4e0 con 0x7f7ba809aa00 2024-01-28T12:56:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f08bffff640 1 -- 172.21.15.31:0/4148111608 >> v1:172.21.15.31:6789/0 conn(0x7f08c006d260 legacy=0x7f08c0098630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f08bffff640 1 -- 172.21.15.31:0/4148111608 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c006de40 con 0x7f08c0098d40 2024-01-28T12:56:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f56ab7fe640 1 -- 172.21.15.31:0/177110586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f56a0009d40 con 0x7f56ac099150 2024-01-28T12:56:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f7bae3c9640 1 --2- 172.21.15.31:0/4011401716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba809ae10 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddc793153c0b346 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f08bffff640 1 --2- 172.21.15.31:0/4148111608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c0098d40 0x7f08c01304a0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665d664fe25bf2b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f56ab7fe640 1 -- 172.21.15.31:0/177110586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f56a000e230 con 0x7f56ac099150 2024-01-28T12:56:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f56ab7fe640 1 -- 172.21.15.31:0/177110586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f56a0009ea0 con 0x7f56ac099150 2024-01-28T12:56:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7fc1dd007640 1 -- 172.21.15.31:0/331663352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc1c8009d40 con 0x7fc1d81363e0 2024-01-28T12:56:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f7bad3c7640 1 -- 172.21.15.31:0/4011401716 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7ba4009dd0 con 0x7f7ba809aa00 2024-01-28T12:56:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f08bf7fe640 1 -- 172.21.15.31:0/4148111608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08ac009ce0 con 0x7f08c0098d40 2024-01-28T12:56:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7fc1dd007640 1 -- 172.21.15.31:0/331663352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc1c800e230 con 0x7fc1d81363e0 2024-01-28T12:56:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7fc1dd007640 1 -- 172.21.15.31:0/331663352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc1c8009ea0 con 0x7fc1d81363e0 2024-01-28T12:56:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f7bad3c7640 1 -- 172.21.15.31:0/4011401716 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7ba400e230 con 0x7f7ba809aa00 2024-01-28T12:56:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f08bf7fe640 1 -- 172.21.15.31:0/4148111608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f08ac009e40 con 0x7f08c0098d40 2024-01-28T12:56:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.058+0000 7f08bf7fe640 1 -- 172.21.15.31:0/4148111608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08ac00e360 con 0x7f08c0098d40 2024-01-28T12:56:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f56b2e64640 1 -- 172.21.15.31:0/177110586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac099150 msgr2=0x7f56ac136630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f56b2e64640 1 --2- 172.21.15.31:0/177110586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac099150 0x7f56ac136630 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f7bad3c7640 1 -- 172.21.15.31:0/4011401716 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7ba400e4d0 con 0x7f7ba809aa00 2024-01-28T12:56:34.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7fc1e026d640 1 -- 172.21.15.31:0/331663352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81363e0 msgr2=0x7fc1d8138800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f56b2e64640 1 -- 172.21.15.31:0/177110586 shutdown_connections 2024-01-28T12:56:34.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f56b2e64640 1 --2- 172.21.15.31:0/177110586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac099150 0x7f56ac136630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f56b2e64640 1 -- 172.21.15.31:0/177110586 >> 172.21.15.31:0/177110586 conn(0x7f56ac09f030 msgr2=0x7f56ac09f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7fc1e026d640 1 --2- 172.21.15.31:0/331663352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81363e0 0x7fc1d8138800 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f7bb062d640 1 -- 172.21.15.31:0/4011401716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 msgr2=0x7f7ba809ae10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f7bb062d640 1 --2- 172.21.15.31:0/4011401716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba809ae10 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f08c7060640 1 -- 172.21.15.31:0/4148111608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c0098d40 msgr2=0x7f08c01304a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f08c7060640 1 --2- 172.21.15.31:0/4148111608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c0098d40 0x7f08c01304a0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7fc1e026d640 1 -- 172.21.15.31:0/331663352 shutdown_connections 2024-01-28T12:56:34.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7fc1e026d640 1 --2- 172.21.15.31:0/331663352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81363e0 0x7fc1d8138800 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7fc1e026d640 1 -- 172.21.15.31:0/331663352 >> 172.21.15.31:0/331663352 conn(0x7fc1d80a6000 msgr2=0x7fc1d80989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f56b2e64640 1 -- 172.21.15.31:0/177110586 shutdown_connections 2024-01-28T12:56:34.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f56b2e64640 1 -- 172.21.15.31:0/177110586 wait complete. 2024-01-28T12:56:34.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f7bb062d640 1 -- 172.21.15.31:0/4011401716 shutdown_connections 2024-01-28T12:56:34.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f7bb062d640 1 --2- 172.21.15.31:0/4011401716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba809ae10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f7bb062d640 1 -- 172.21.15.31:0/4011401716 >> 172.21.15.31:0/4011401716 conn(0x7f7ba80a3500 msgr2=0x7f7ba80a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f08c7060640 1 -- 172.21.15.31:0/4148111608 shutdown_connections 2024-01-28T12:56:34.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f08c7060640 1 --2- 172.21.15.31:0/4148111608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c0098d40 0x7f08c01304a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f08c7060640 1 -- 172.21.15.31:0/4148111608 >> 172.21.15.31:0/4148111608 conn(0x7f08c009f030 msgr2=0x7f08c009f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7fc1e026d640 1 -- 172.21.15.31:0/331663352 shutdown_connections 2024-01-28T12:56:34.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f08c7060640 1 -- 172.21.15.31:0/4148111608 shutdown_connections 2024-01-28T12:56:34.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7fc1e026d640 1 -- 172.21.15.31:0/331663352 wait complete. 2024-01-28T12:56:34.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.059+0000 7f08c7060640 1 -- 172.21.15.31:0/4148111608 wait complete. 2024-01-28T12:56:34.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.060+0000 7f7bb062d640 1 -- 172.21.15.31:0/4011401716 shutdown_connections 2024-01-28T12:56:34.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.060+0000 7f7bb062d640 1 -- 172.21.15.31:0/4011401716 wait complete. 2024-01-28T12:56:34.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.060+0000 7f56b2e64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.060+0000 7f08c7060640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.060+0000 7f7bb062d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.060+0000 7fc1e026d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f08c7060640 1 Processor -- start 2024-01-28T12:56:34.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f08c7060640 1 -- start start 2024-01-28T12:56:34.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f56b2e64640 1 Processor -- start 2024-01-28T12:56:34.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f56b2e64640 1 -- start start 2024-01-28T12:56:34.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7fc1e026d640 1 Processor -- start 2024-01-28T12:56:34.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f7bb062d640 1 Processor -- start 2024-01-28T12:56:34.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7fc1e026d640 1 -- start start 2024-01-28T12:56:34.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f7bb062d640 1 -- start start 2024-01-28T12:56:34.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f08c7060640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c006d260 0x7f08c006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.061+0000 7f08c7060640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0071950 con 0x7f08c006d260 2024-01-28T12:56:34.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f08c4dfc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c006d260 0x7f08c006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f08c4dfc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c006d260 0x7f08c006e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57008/0 (socket says 172.21.15.31:57008) 2024-01-28T12:56:34.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f08c4dfc640 1 -- 172.21.15.31:0/40705107 learned_addr learned my addr 172.21.15.31:0/40705107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f56b2e64640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac098630 0x7f56ac12f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f56b2e64640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac12e4b0 con 0x7f56ac098630 2024-01-28T12:56:34.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f08c4dfc640 1 -- 172.21.15.31:0/40705107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c0071b40 con 0x7f08c006d260 2024-01-28T12:56:34.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f56b0c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac098630 0x7f56ac12f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f56b0c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac098630 0x7f56ac12f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57022/0 (socket says 172.21.15.31:57022) 2024-01-28T12:56:34.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f56b0c00640 1 -- 172.21.15.31:0/3384030407 learned_addr learned my addr 172.21.15.31:0/3384030407 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f7bb062d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba811c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f56b0c00640 1 -- 172.21.15.31:0/3384030407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac12da00 con 0x7f56ac098630 2024-01-28T12:56:34.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f7bb062d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba811ff30 con 0x7f7ba809aa00 2024-01-28T12:56:34.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.062+0000 7f08c4dfc640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c006d260 0x7f08c006e980 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f56b0c00640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac098630 0x7f56ac12f230 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f569800b8b0 con 0x7f56ac098630 2024-01-28T12:56:34.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f7bae3c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba811c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f7bae3c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba811c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57026/0 (socket says 172.21.15.31:57026) 2024-01-28T12:56:34.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f7bae3c9640 1 -- 172.21.15.31:0/1680732207 learned_addr learned my addr 172.21.15.31:0/1680732207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f569800ba10 con 0x7f56ac098630 2024-01-28T12:56:34.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f56b2e64640 1 -- 172.21.15.31:0/3384030407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac12dc30 con 0x7f56ac098630 2024-01-28T12:56:34.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08b4004bd0 con 0x7f08c006d260 2024-01-28T12:56:34.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f08b4004d30 con 0x7f08c006d260 2024-01-28T12:56:34.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c00705f0 con 0x7f08c006d260 2024-01-28T12:56:34.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c0089f80 con 0x7f08c006d260 2024-01-28T12:56:34.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f569800bcb0 con 0x7f56ac098630 2024-01-28T12:56:34.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08b4004bd0 con 0x7f08c006d260 2024-01-28T12:56:34.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f56b2e64640 1 -- 172.21.15.31:0/3384030407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac12ef20 con 0x7f56ac098630 2024-01-28T12:56:34.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7fc1e026d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81300e0 0x7fc1d811bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7fc1e026d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d809c2c0 con 0x7fc1d81300e0 2024-01-28T12:56:34.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.063+0000 7f7bae3c9640 1 -- 172.21.15.31:0/1680732207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8120430 con 0x7f7ba809aa00 2024-01-28T12:56:34.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7fc1de009640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81300e0 0x7fc1d811bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7fc1de009640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81300e0 0x7fc1d811bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57028/0 (socket says 172.21.15.31:57028) 2024-01-28T12:56:34.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08c0068130 con 0x7f08c006d260 2024-01-28T12:56:34.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7fc1de009640 1 -- 172.21.15.31:0/3037857829 learned_addr learned my addr 172.21.15.31:0/3037857829 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f7bae3c9640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba811c840 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f56b2e64640 1 -- 172.21.15.31:0/3384030407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f56ac09af60 con 0x7f56ac098630 2024-01-28T12:56:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7ba4014410 con 0x7f7ba809aa00 2024-01-28T12:56:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7ba4014570 con 0x7f7ba809aa00 2024-01-28T12:56:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7ba4014810 con 0x7f7ba809aa00 2024-01-28T12:56:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba811ed30 con 0x7f7ba809aa00 2024-01-28T12:56:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba8130bb0 con 0x7f7ba809aa00 2024-01-28T12:56:34.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.064+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f5698011380 con 0x7f56ac098630 2024-01-28T12:56:34.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7fc1de009640 1 -- 172.21.15.31:0/3037857829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d811ece0 con 0x7fc1d81300e0 2024-01-28T12:56:34.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7fc1de009640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81300e0 0x7fc1d811bbe0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f56a97fa640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5690037a90 0x7f5690039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f56980476f0 con 0x7f56ac098630 2024-01-28T12:56:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f56a97fa640 1 --2- 172.21.15.31:0/3384030407 >> v2:172.21.15.31:6800/507996357 conn(0x7f569003ca30 0x7f569003eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 --> v2:172.21.15.31:6800/507996357 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f569003f550 con 0x7f569003ca30 2024-01-28T12:56:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7f5698011680 con 0x7f56ac098630 2024-01-28T12:56:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ba809d780 con 0x7f7ba809aa00 2024-01-28T12:56:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f56abfff640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5690037a90 0x7f5690039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc1d400b880 con 0x7fc1d81300e0 2024-01-28T12:56:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f08b4007120 con 0x7f08c006d260 2024-01-28T12:56:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc1d400b9e0 con 0x7fc1d81300e0 2024-01-28T12:56:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d806af40 con 0x7fc1d81300e0 2024-01-28T12:56:34.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.065+0000 7f56b1401640 1 --2- 172.21.15.31:0/3384030407 >> v2:172.21.15.31:6800/507996357 conn(0x7f569003ca30 0x7f569003eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-28T12:56:34.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc1d400bc80 con 0x7fc1d81300e0 2024-01-28T12:56:34.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d811d820 con 0x7fc1d81300e0 2024-01-28T12:56:34.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f08bd7fa640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f08a4037a30 0x7f08a4039f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f08b40470d0 con 0x7f08c006d260 2024-01-28T12:56:34.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f08bd7fa640 1 --2- 172.21.15.31:0/40705107 >> v2:172.21.15.137:6802/3189673583 conn(0x7f08a403c9d0 0x7f08a403ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 --> v2:172.21.15.137:6802/3189673583 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08a403f4f0 con 0x7f08a403c9d0 2024-01-28T12:56:34.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7f08b40473f0 con 0x7f08c006d260 2024-01-28T12:56:34.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f7ba40149c0 con 0x7f7ba809aa00 2024-01-28T12:56:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f56abfff640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5690037a90 0x7f5690039fa0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f7b9effd640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7b8c037a90 0x7f7b8c039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7ba4019040 con 0x7f7ba809aa00 2024-01-28T12:56:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f7b9effd640 1 --2- 172.21.15.31:0/1680732207 >> v2:172.21.15.31:6803/2734761187 conn(0x7f7b8c03ca30 0x7f7b8c03eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 --> v2:172.21.15.31:6803/2734761187 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b8c03f550 con 0x7f7b8c03ca30 2024-01-28T12:56:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f7badbc8640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7b8c037a90 0x7f7b8c039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7f7ba404ab20 con 0x7f7ba809aa00 2024-01-28T12:56:34.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f08bffff640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f08a4037a30 0x7f08a4039f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.066+0000 7f08bffff640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f08a4037a30 0x7f08a4039f40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7fc1d4011380 con 0x7fc1d81300e0 2024-01-28T12:56:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7f08c55fd640 1 --2- 172.21.15.31:0/40705107 >> v2:172.21.15.137:6802/3189673583 conn(0x7f08a403c9d0 0x7f08a403ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-28T12:56:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1d809c780 con 0x7fc1d81300e0 2024-01-28T12:56:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1ceffd640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fc1bc037a40 0x7fc1bc039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc1d40477b0 con 0x7fc1d81300e0 2024-01-28T12:56:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7f7badbc8640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7b8c037a90 0x7f7b8c039fa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1ceffd640 1 --2- 172.21.15.31:0/3037857829 >> v2:172.21.15.137:6804/964334066 conn(0x7fc1bc03c9e0 0x7fc1bc03ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 --> v2:172.21.15.137:6804/964334066 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1bc03f500 con 0x7fc1bc03c9e0 2024-01-28T12:56:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7fc1d4049ca0 con 0x7fc1d81300e0 2024-01-28T12:56:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1dd808640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fc1bc037a40 0x7fc1bc039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.067+0000 7fc1dd808640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fc1bc037a40 0x7fc1bc039f50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.068+0000 7f56b1401640 1 --2- 172.21.15.31:0/3384030407 >> v2:172.21.15.31:6800/507996357 conn(0x7f569003ca30 0x7f569003eec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.069+0000 7f08c55fd640 1 --2- 172.21.15.31:0/40705107 >> v2:172.21.15.137:6802/3189673583 conn(0x7f08a403c9d0 0x7f08a403ee60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.069+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== osd.0 v2:172.21.15.31:6800/507996357 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f569003f550 con 0x7f569003ca30 2024-01-28T12:56:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.070+0000 7f7baebca640 1 --2- 172.21.15.31:0/1680732207 >> v2:172.21.15.31:6803/2734761187 conn(0x7f7b8c03ca30 0x7f7b8c03eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-28T12:56:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.070+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== osd.3 v2:172.21.15.137:6802/3189673583 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f08a403f4f0 con 0x7f08a403c9d0 2024-01-28T12:56:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.071+0000 7fc1de80a640 1 --2- 172.21.15.31:0/3037857829 >> v2:172.21.15.137:6804/964334066 conn(0x7fc1bc03c9e0 0x7fc1bc03ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-28T12:56:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.072+0000 7f7baebca640 1 --2- 172.21.15.31:0/1680732207 >> v2:172.21.15.31:6803/2734761187 conn(0x7f7b8c03ca30 0x7f7b8c03eec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.072+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== osd.1 v2:172.21.15.31:6803/2734761187 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f7b8c03f550 con 0x7f7b8c03ca30 2024-01-28T12:56:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.073+0000 7fc1de80a640 1 --2- 172.21.15.31:0/3037857829 >> v2:172.21.15.137:6804/964334066 conn(0x7fc1bc03c9e0 0x7fc1bc03ee70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.074+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== osd.2 v2:172.21.15.137:6804/964334066 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7fc1bc03f500 con 0x7fc1bc03c9e0 2024-01-28T12:56:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.075+0000 7f56b2e64640 1 -- 172.21.15.31:0/3384030407 --> v2:172.21.15.31:6800/507996357 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f56ac12c460 con 0x7f569003ca30 2024-01-28T12:56:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.076+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 --> v2:172.21.15.31:6803/2734761187 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ba811f5e0 con 0x7f7b8c03ca30 2024-01-28T12:56:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.076+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 --> v2:172.21.15.137:6802/3189673583 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08c009b3c0 con 0x7f08a403c9d0 2024-01-28T12:56:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.077+0000 7f56a97fa640 1 -- 172.21.15.31:0/3384030407 <== osd.0 v2:172.21.15.31:6800/507996357 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f56ac12c460 con 0x7f569003ca30 2024-01-28T12:56:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.077+0000 7f5696ffd640 1 -- 172.21.15.31:0/3384030407 >> v2:172.21.15.31:6800/507996357 conn(0x7f569003ca30 msgr2=0x7f569003eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 --2- 172.21.15.31:0/3384030407 >> v2:172.21.15.31:6800/507996357 conn(0x7f569003ca30 0x7f569003eec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08bd7fa640 1 -- 172.21.15.31:0/40705107 <== osd.3 v2:172.21.15.137:6802/3189673583 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08b800a820 con 0x7f08a403c9d0 2024-01-28T12:56:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 -- 172.21.15.31:0/3384030407 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5690037a90 msgr2=0x7f5690039fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 >> v2:172.21.15.137:6802/3189673583 conn(0x7f08a403c9d0 msgr2=0x7f08a403ee60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 --2- 172.21.15.31:0/40705107 >> v2:172.21.15.137:6802/3189673583 conn(0x7f08a403c9d0 0x7f08a403ee60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5690037a90 0x7f5690039fa0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 -- 172.21.15.31:0/3384030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac098630 msgr2=0x7f56ac12f230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac098630 0x7f56ac12f230 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f08a4037a30 msgr2=0x7f08a4039f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f08a4037a30 0x7f08a4039f40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c006d260 msgr2=0x7f08c006e980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c006d260 0x7f08c006e980 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 -- 172.21.15.31:0/3384030407 shutdown_connections 2024-01-28T12:56:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 --2- 172.21.15.31:0/3384030407 >> v2:172.21.15.31:6800/507996357 conn(0x7f569003ca30 0x7f569003eec0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f5690037a90 0x7f5690039fa0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 --2- 172.21.15.31:0/3384030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ac098630 0x7f56ac12f230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 -- 172.21.15.31:0/3384030407 >> 172.21.15.31:0/3384030407 conn(0x7f56ac09f030 msgr2=0x7f56ac1b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 shutdown_connections 2024-01-28T12:56:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 --2- 172.21.15.31:0/40705107 >> v2:172.21.15.137:6802/3189673583 conn(0x7f08a403c9d0 0x7f08a403ee60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f08a4037a30 0x7f08a4039f40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 --2- 172.21.15.31:0/40705107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c006d260 0x7f08c006e980 crc :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 >> 172.21.15.31:0/40705107 conn(0x7f08c009f030 msgr2=0x7f08c009ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.078+0000 7f5696ffd640 1 -- 172.21.15.31:0/3384030407 shutdown_connections 2024-01-28T12:56:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f5696ffd640 1 -- 172.21.15.31:0/3384030407 wait complete. 2024-01-28T12:56:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 shutdown_connections 2024-01-28T12:56:34.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f08c7060640 1 -- 172.21.15.31:0/40705107 wait complete. 2024-01-28T12:56:34.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f7b9effd640 1 -- 172.21.15.31:0/1680732207 <== osd.1 v2:172.21.15.31:6803/2734761187 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7ba811f5e0 con 0x7f7b8c03ca30 2024-01-28T12:56:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 >> v2:172.21.15.31:6803/2734761187 conn(0x7f7b8c03ca30 msgr2=0x7f7b8c03eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f7bb062d640 1 --2- 172.21.15.31:0/1680732207 >> v2:172.21.15.31:6803/2734761187 conn(0x7f7b8c03ca30 0x7f7b8c03eec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7b8c037a90 msgr2=0x7f7b8c039fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f7bb062d640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7b8c037a90 0x7f7b8c039fa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.079+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 msgr2=0x7f7ba811c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7f7bb062d640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba811c840 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 shutdown_connections 2024-01-28T12:56:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7f7bb062d640 1 --2- 172.21.15.31:0/1680732207 >> v2:172.21.15.31:6803/2734761187 conn(0x7f7b8c03ca30 0x7f7b8c03eec0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7f7bb062d640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7b8c037a90 0x7f7b8c039fa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7f7bb062d640 1 --2- 172.21.15.31:0/1680732207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba809aa00 0x7f7ba811c840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 >> 172.21.15.31:0/1680732207 conn(0x7f7ba80a3500 msgr2=0x7f7ba8098a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 shutdown_connections 2024-01-28T12:56:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.080+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 --> v2:172.21.15.137:6804/964334066 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1d8075440 con 0x7fc1bc03c9e0 2024-01-28T12:56:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.081+0000 7f7bb062d640 1 -- 172.21.15.31:0/1680732207 wait complete. 2024-01-28T12:56:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.081+0000 7fc1ceffd640 1 -- 172.21.15.31:0/3037857829 <== osd.2 v2:172.21.15.137:6804/964334066 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc1d8075440 con 0x7fc1bc03c9e0 2024-01-28T12:56:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 >> v2:172.21.15.137:6804/964334066 conn(0x7fc1bc03c9e0 msgr2=0x7fc1bc03ee70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 --2- 172.21.15.31:0/3037857829 >> v2:172.21.15.137:6804/964334066 conn(0x7fc1bc03c9e0 0x7fc1bc03ee70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fc1bc037a40 msgr2=0x7fc1bc039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fc1bc037a40 0x7fc1bc039f50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81300e0 msgr2=0x7fc1d811bbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81300e0 0x7fc1d811bbe0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 shutdown_connections 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 --2- 172.21.15.31:0/3037857829 >> v2:172.21.15.137:6804/964334066 conn(0x7fc1bc03c9e0 0x7fc1bc03ee70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7fc1bc037a40 0x7fc1bc039f50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 --2- 172.21.15.31:0/3037857829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1d81300e0 0x7fc1d811bbe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 >> 172.21.15.31:0/3037857829 conn(0x7fc1d80a6000 msgr2=0x7fc1d81bb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 shutdown_connections 2024-01-28T12:56:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.082+0000 7fc1e026d640 1 -- 172.21.15.31:0/3037857829 wait complete. 2024-01-28T12:56:34.111 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-28T12:56:34.112 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-28T12:56:34.113 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-28T12:56:34.114 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-28T12:56:34.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.267+0000 7ff984c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.267+0000 7f21e1c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.267+0000 7f7e6ff1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.271+0000 7f7e6ff1e640 1 Processor -- start 2024-01-28T12:56:34.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.271+0000 7ff984c6d640 1 Processor -- start 2024-01-28T12:56:34.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7ff984c6d640 1 -- start start 2024-01-28T12:56:34.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.267+0000 7f4d2155e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f21e1c2d640 1 Processor -- start 2024-01-28T12:56:34.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f7e6ff1e640 1 -- start start 2024-01-28T12:56:34.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f21e1c2d640 1 -- start start 2024-01-28T12:56:34.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f4d2155e640 1 Processor -- start 2024-01-28T12:56:34.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f4d2155e640 1 -- start start 2024-01-28T12:56:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f7e6ff1e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e6809aaf0 0x7f7e681354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f7e6ff1e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6806f390 con 0x7f7e68099fd0 2024-01-28T12:56:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f7e6ff1e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e680987c0 con 0x7f7e6809aaf0 2024-01-28T12:56:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff984c6d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff980136650 0x7ff980138a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.272+0000 7f21e1c2d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21dc12fad0 0x7f21dc12fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff984c6d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98006c170 con 0x7ff980133830 2024-01-28T12:56:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff984c6d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98006d470 con 0x7ff980136650 2024-01-28T12:56:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f21e1c2d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc06e7f0 con 0x7f21dc098630 2024-01-28T12:56:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f21e1c2d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc06dcd0 con 0x7f21dc12fad0 2024-01-28T12:56:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff97dd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff980136650 0x7ff980138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f21db7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21dc12fad0 0x7f21dc12fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff97e59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff980133830 0x7ff980133c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60956/0 (socket says 172.21.15.31:60956) 2024-01-28T12:56:34.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff97e59c640 1 -- 172.21.15.31:0/2125028765 learned_addr learned my addr 172.21.15.31:0/2125028765 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:34.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f21db7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21dc12fad0 0x7f21dc12fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57042/0 (socket says 172.21.15.31:57042) 2024-01-28T12:56:34.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f21db7fe640 1 -- 172.21.15.31:0/2911320033 learned_addr learned my addr 172.21.15.31:0/2911320033 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f7e6d4b9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e6809aaf0 0x7f7e681354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f7e6dcba640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7e68099fd0 0x7f7e6809a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60930/0 (socket says 172.21.15.31:60930) 2024-01-28T12:56:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f7e6d4b9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e6809aaf0 0x7f7e681354d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57038/0 (socket says 172.21.15.31:57038) 2024-01-28T12:56:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f7e6dcba640 1 -- 172.21.15.31:0/1639560973 learned_addr learned my addr 172.21.15.31:0/1639560973 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff97dd9b640 1 -- 172.21.15.31:0/2125028765 >> v1:172.21.15.31:6789/0 conn(0x7ff980133830 legacy=0x7ff980133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff97dd9b640 1 -- 172.21.15.31:0/2125028765 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9801333e0 con 0x7ff980136650 2024-01-28T12:56:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f21db7fe640 1 -- 172.21.15.31:0/2911320033 >> v1:172.21.15.31:6789/0 conn(0x7f21dc098630 legacy=0x7f21dc098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f21db7fe640 1 -- 172.21.15.31:0/2911320033 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc12f6a0 con 0x7f21dc12fad0 2024-01-28T12:56:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7ff97dd9b640 1 --2- 172.21.15.31:0/2125028765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980136650 0x7ff980138a70 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cd1bcf6c9e62 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f4d2155e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d1c09a560 0x7f4d1c135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f7e6d4b9640 1 -- 172.21.15.31:0/1639560973 >> v1:172.21.15.31:6789/0 conn(0x7f7e68099fd0 legacy=0x7f7e6809a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-28T12:56:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.273+0000 7f7e6d4b9640 1 -- 172.21.15.31:0/1639560973 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6806f080 con 0x7f7e6809aaf0 2024-01-28T12:56:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d2155e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c09c480 con 0x7f4d1c09dc30 2024-01-28T12:56:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d2155e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c09a970 con 0x7f4d1c09a560 2024-01-28T12:56:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d1affd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d1c09dc30 0x7f4d1c099e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60966/0 (socket says 172.21.15.31:60966) 2024-01-28T12:56:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d1affd640 1 -- 172.21.15.31:0/1070044351 learned_addr learned my addr 172.21.15.31:0/1070044351 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f21db7fe640 1 --2- 172.21.15.31:0/2911320033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc12fad0 0x7f21dc12fee0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b07b510c1e36de3 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d1a7fc640 1 --2- 172.21.15.31:0/1070044351 >> v2:172.21.15.31:3300/0 conn(0x7f4d1c09a560 0x7f4d1c135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7ff97d59a640 1 -- 172.21.15.31:0/2125028765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff974009ce0 con 0x7ff980136650 2024-01-28T12:56:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f21da7fc640 1 -- 172.21.15.31:0/2911320033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f21cc009dd0 con 0x7f21dc12fad0 2024-01-28T12:56:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7ff97d59a640 1 -- 172.21.15.31:0/2125028765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff974009e40 con 0x7ff980136650 2024-01-28T12:56:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f21da7fc640 1 -- 172.21.15.31:0/2911320033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f21cc00e230 con 0x7f21dc12fad0 2024-01-28T12:56:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f21da7fc640 1 -- 172.21.15.31:0/2911320033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f21cc00e4d0 con 0x7f21dc12fad0 2024-01-28T12:56:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f7e6d4b9640 1 --2- 172.21.15.31:0/1639560973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e6809aaf0 0x7f7e681354d0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02099a6183b7c server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d19ffb640 1 -- 172.21.15.31:0/1070044351 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787171895 0 0) 0x7f4d1c09c480 con 0x7f4d1c09dc30 2024-01-28T12:56:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d19ffb640 1 -- 172.21.15.31:0/1070044351 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d040034f0 con 0x7f4d1c09dc30 2024-01-28T12:56:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7ff97d59a640 1 -- 172.21.15.31:0/2125028765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff97400e360 con 0x7ff980136650 2024-01-28T12:56:34.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d1a7fc640 1 -- 172.21.15.31:0/1070044351 >> v1:172.21.15.31:6789/0 conn(0x7f4d1c09dc30 legacy=0x7f4d1c099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d1a7fc640 1 -- 172.21.15.31:0/1070044351 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c099640 con 0x7f4d1c09a560 2024-01-28T12:56:34.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f4d1a7fc640 1 --2- 172.21.15.31:0/1070044351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c135130 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1703806371e06ee server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f7e6ccb8640 1 -- 172.21.15.31:0/1639560973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e58009c50 con 0x7f7e6809aaf0 2024-01-28T12:56:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f7e6ccb8640 1 -- 172.21.15.31:0/1639560973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7e58009db0 con 0x7f7e6809aaf0 2024-01-28T12:56:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.274+0000 7f7e6ccb8640 1 -- 172.21.15.31:0/1639560973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e5800e360 con 0x7f7e6809aaf0 2024-01-28T12:56:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f4d19ffb640 1 -- 172.21.15.31:0/1070044351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d10009ce0 con 0x7f4d1c09a560 2024-01-28T12:56:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f4d19ffb640 1 -- 172.21.15.31:0/1070044351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4d10009e40 con 0x7f4d1c09a560 2024-01-28T12:56:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f4d19ffb640 1 -- 172.21.15.31:0/1070044351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d10010570 con 0x7f4d1c09a560 2024-01-28T12:56:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 1 -- 172.21.15.31:0/2125028765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980136650 msgr2=0x7ff980138a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 1 --2- 172.21.15.31:0/2125028765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980136650 0x7ff980138a70 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 1 -- 172.21.15.31:0/2125028765 shutdown_connections 2024-01-28T12:56:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 1 --2- 172.21.15.31:0/2125028765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980136650 0x7ff980138a70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 1 -- 172.21.15.31:0/2125028765 >> 172.21.15.31:0/2125028765 conn(0x7ff980098b90 msgr2=0x7ff980098fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 1 -- 172.21.15.31:0/2911320033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc12fad0 msgr2=0x7f21dc12fee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 1 --2- 172.21.15.31:0/2911320033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc12fad0 0x7f21dc12fee0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 1 -- 172.21.15.31:0/2125028765 shutdown_connections 2024-01-28T12:56:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 1 -- 172.21.15.31:0/2125028765 wait complete. 2024-01-28T12:56:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 1 -- 172.21.15.31:0/2911320033 shutdown_connections 2024-01-28T12:56:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 1 --2- 172.21.15.31:0/2911320033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc12fad0 0x7f21dc12fee0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 1 -- 172.21.15.31:0/2911320033 >> 172.21.15.31:0/2911320033 conn(0x7f21dc09f030 msgr2=0x7f21dc09f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 1 -- 172.21.15.31:0/2911320033 shutdown_connections 2024-01-28T12:56:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 1 -- 172.21.15.31:0/2911320033 wait complete. 2024-01-28T12:56:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7ff984c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f21e1c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f4d2155e640 1 -- 172.21.15.31:0/1070044351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 msgr2=0x7f4d1c135130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.275+0000 7f4d2155e640 1 --2- 172.21.15.31:0/1070044351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c135130 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/1639560973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e6809aaf0 msgr2=0x7f7e681354d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 1 --2- 172.21.15.31:0/1639560973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e6809aaf0 0x7f7e681354d0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/1639560973 shutdown_connections 2024-01-28T12:56:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 1 --2- 172.21.15.31:0/1639560973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e6809aaf0 0x7f7e681354d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f4d2155e640 1 -- 172.21.15.31:0/1070044351 shutdown_connections 2024-01-28T12:56:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f4d2155e640 1 --2- 172.21.15.31:0/1070044351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c135130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/1639560973 >> 172.21.15.31:0/1639560973 conn(0x7f7e6809f030 msgr2=0x7f7e6809f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f4d2155e640 1 -- 172.21.15.31:0/1070044351 >> 172.21.15.31:0/1070044351 conn(0x7f4d1c09ede0 msgr2=0x7f4d1c09f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f4d2155e640 1 -- 172.21.15.31:0/1070044351 shutdown_connections 2024-01-28T12:56:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f4d2155e640 1 -- 172.21.15.31:0/1070044351 wait complete. 2024-01-28T12:56:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/1639560973 shutdown_connections 2024-01-28T12:56:34.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/1639560973 wait complete. 2024-01-28T12:56:34.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f4d2155e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f7e6ff1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7f21e1c2d640 1 Processor -- start 2024-01-28T12:56:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7ff984c6d640 1 Processor -- start 2024-01-28T12:56:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.276+0000 7ff984c6d640 1 -- start start 2024-01-28T12:56:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f21e1c2d640 1 -- start start 2024-01-28T12:56:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f4d2155e640 1 Processor -- start 2024-01-28T12:56:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f4d2155e640 1 -- start start 2024-01-28T12:56:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f21e1c2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc098630 0x7f21dc08e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f21e1c2d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc08bc40 con 0x7f21dc098630 2024-01-28T12:56:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f7e6ff1e640 1 Processor -- start 2024-01-28T12:56:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7ff984c6d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980133830 0x7ff9801c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f7e6ff1e640 1 -- start start 2024-01-28T12:56:34.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7ff984c6d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff980121770 con 0x7ff980133830 2024-01-28T12:56:34.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc098630 0x7f21dc08e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc098630 0x7f21dc08e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57054/0 (socket says 172.21.15.31:57054) 2024-01-28T12:56:34.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.277+0000 7f21db7fe640 1 -- 172.21.15.31:0/507152371 learned_addr learned my addr 172.21.15.31:0/507152371 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7ff97e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980133830 0x7ff9801c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7ff97e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980133830 0x7ff9801c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57060/0 (socket says 172.21.15.31:57060) 2024-01-28T12:56:34.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7ff97e59c640 1 -- 172.21.15.31:0/301351880 learned_addr learned my addr 172.21.15.31:0/301351880 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f21db7fe640 1 -- 172.21.15.31:0/507152371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc08de40 con 0x7f21dc098630 2024-01-28T12:56:34.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f21db7fe640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc098630 0x7f21dc08e450 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7ff97e59c640 1 -- 172.21.15.31:0/301351880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98006e020 con 0x7ff980133830 2024-01-28T12:56:34.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f7e6ff1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e68099fd0 0x7f7e681b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f7e6ff1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681b8da0 con 0x7f7e68099fd0 2024-01-28T12:56:34.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f4d2155e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c11d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f7e6dcba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e68099fd0 0x7f7e681b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f21c3fff640 1 -- 172.21.15.31:0/507152371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f21cc014750 con 0x7f21dc098630 2024-01-28T12:56:34.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f7e6dcba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e68099fd0 0x7f7e681b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57076/0 (socket says 172.21.15.31:57076) 2024-01-28T12:56:34.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f4d2155e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c06d070 con 0x7f4d1c09a560 2024-01-28T12:56:34.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc06bf20 con 0x7f21dc098630 2024-01-28T12:56:34.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f7e6dcba640 1 -- 172.21.15.31:0/2089136352 learned_addr learned my addr 172.21.15.31:0/2089136352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7ff97e59c640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980133830 0x7ff9801c4e50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc08da80 con 0x7f21dc098630 2024-01-28T12:56:34.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f21c3fff640 1 -- 172.21.15.31:0/507152371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f21cc0148b0 con 0x7f21dc098630 2024-01-28T12:56:34.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f21c3fff640 1 -- 172.21.15.31:0/507152371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f21cc014b50 con 0x7f21dc098630 2024-01-28T12:56:34.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f4d1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c11d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f4d1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c11d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57084/0 (socket says 172.21.15.31:57084) 2024-01-28T12:56:34.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7f4d1affd640 1 -- 172.21.15.31:0/2767401364 learned_addr learned my addr 172.21.15.31:0/2767401364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.278+0000 7ff96effd640 1 -- 172.21.15.31:0/301351880 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff9700059d0 con 0x7ff980133830 2024-01-28T12:56:34.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7f7e6dcba640 1 -- 172.21.15.31:0/2089136352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e681b83b0 con 0x7f7e68099fd0 2024-01-28T12:56:34.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7ff984c6d640 1 -- 172.21.15.31:0/301351880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98011f210 con 0x7ff980133830 2024-01-28T12:56:34.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7ff96effd640 1 -- 172.21.15.31:0/301351880 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff970005b30 con 0x7ff980133830 2024-01-28T12:56:34.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7f7e6dcba640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e68099fd0 0x7f7e681b54e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7f4d1affd640 1 -- 172.21.15.31:0/2767401364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c11fd70 con 0x7f4d1c09a560 2024-01-28T12:56:34.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7f4d1affd640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c11d860 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7f7e5e7fc640 1 -- 172.21.15.31:0/2089136352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e6400b860 con 0x7f7e68099fd0 2024-01-28T12:56:34.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e681b8b00 con 0x7f7e68099fd0 2024-01-28T12:56:34.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7ff96effd640 1 -- 172.21.15.31:0/301351880 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff970005dd0 con 0x7ff980133830 2024-01-28T12:56:34.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.279+0000 7ff984c6d640 1 -- 172.21.15.31:0/301351880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff980120170 con 0x7ff980133830 2024-01-28T12:56:34.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7ff96effd640 1 -- 172.21.15.31:0/301351880 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7ff9700070b0 con 0x7ff980133830 2024-01-28T12:56:34.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7ff96effd640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7ff960037a40 0x7ff960039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc005050 con 0x7f21dc098630 2024-01-28T12:56:34.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e681b6810 con 0x7f7e68099fd0 2024-01-28T12:56:34.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f4d037fe640 1 -- 172.21.15.31:0/2767401364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d0c0059d0 con 0x7f4d1c09a560 2024-01-28T12:56:34.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c121860 con 0x7f4d1c09a560 2024-01-28T12:56:34.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff958005050 con 0x7ff980133830 2024-01-28T12:56:34.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f4d037fe640 1 -- 172.21.15.31:0/2767401364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4d0c005b30 con 0x7f4d1c09a560 2024-01-28T12:56:34.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c11f370 con 0x7f4d1c09a560 2024-01-28T12:56:34.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7ff97dd9b640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7ff960037a40 0x7ff960039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f7e5e7fc640 1 -- 172.21.15.31:0/2089136352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7e6400b9c0 con 0x7f7e68099fd0 2024-01-28T12:56:34.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.280+0000 7f7e5e7fc640 1 -- 172.21.15.31:0/2089136352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e6400bc60 con 0x7f7e68099fd0 2024-01-28T12:56:34.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7f21c3fff640 1 -- 172.21.15.31:0/507152371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f21cc013400 con 0x7f21dc098630 2024-01-28T12:56:34.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7f21c3fff640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f21c8037a90 0x7f21c8039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7f4d037fe640 1 -- 172.21.15.31:0/2767401364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d0c0059d0 con 0x7f4d1c09a560 2024-01-28T12:56:34.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7f21c3fff640 1 -- 172.21.15.31:0/507152371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f21cc019040 con 0x7f21dc098630 2024-01-28T12:56:34.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c075440 con 0x7f4d1c09a560 2024-01-28T12:56:34.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7f21daffd640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f21c8037a90 0x7f21c8039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e50005050 con 0x7f7e68099fd0 2024-01-28T12:56:34.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.281+0000 7ff97dd9b640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7ff960037a40 0x7ff960039f50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.282+0000 7f21daffd640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f21c8037a90 0x7f21c8039fa0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.285+0000 7f21c3fff640 1 -- 172.21.15.31:0/507152371 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f21cc00fa90 con 0x7f21dc098630 2024-01-28T12:56:34.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.285+0000 7ff96effd640 1 -- 172.21.15.31:0/301351880 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ff970009650 con 0x7ff980133830 2024-01-28T12:56:34.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.286+0000 7f7e5e7fc640 1 -- 172.21.15.31:0/2089136352 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f7e640123f0 con 0x7f7e68099fd0 2024-01-28T12:56:34.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.286+0000 7f7e5e7fc640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7e4c037a40 0x7f7e4c039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.286+0000 7f4d037fe640 1 -- 172.21.15.31:0/2767401364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f4d0c005d40 con 0x7f4d1c09a560 2024-01-28T12:56:34.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.286+0000 7f4d037fe640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f4d08037a30 0x7f4d08039f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.286+0000 7f7e6d4b9640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7e4c037a40 0x7f7e4c039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.286+0000 7f4d1a7fc640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f4d08037a30 0x7f4d08039f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.287+0000 7f7e6d4b9640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7e4c037a40 0x7f7e4c039f50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.287+0000 7f4d1a7fc640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f4d08037a30 0x7f4d08039f40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.294+0000 7f4d037fe640 1 -- 172.21.15.31:0/2767401364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4d0c00f7f0 con 0x7f4d1c09a560 2024-01-28T12:56:34.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.294+0000 7ff96effd640 1 -- 172.21.15.31:0/301351880 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7ff970009470 con 0x7ff980133830 2024-01-28T12:56:34.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.294+0000 7f7e5e7fc640 1 -- 172.21.15.31:0/2089136352 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7e640047b0 con 0x7f7e68099fd0 2024-01-28T12:56:34.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.304+0000 7f7e5e7fc640 1 -- 172.21.15.31:0/2089136352 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f7e6400ede0 con 0x7f7e68099fd0 2024-01-28T12:56:34.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.313+0000 7f4d037fe640 1 -- 172.21.15.31:0/2767401364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f4d0c00a300 con 0x7f4d1c09a560 2024-01-28T12:56:34.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.473+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff9580055d0 con 0x7ff980133830 2024-01-28T12:56:34.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.474+0000 7ff96effd640 1 -- 172.21.15.31:0/301351880 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7ff970048040 con 0x7ff980133830 2024-01-28T12:56:34.472 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7ff960037a40 msgr2=0x7ff960039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7ff960037a40 0x7ff960039f50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980133830 msgr2=0x7ff9801c4e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980133830 0x7ff9801c4e50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 shutdown_connections 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7ff960037a40 0x7ff960039f50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 --2- 172.21.15.31:0/301351880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980133830 0x7ff9801c4e50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 >> 172.21.15.31:0/301351880 conn(0x7ff980098b90 msgr2=0x7ff980131d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 shutdown_connections 2024-01-28T12:56:34.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.476+0000 7ff96cff9640 1 -- 172.21.15.31:0/301351880 wait complete. 2024-01-28T12:56:34.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.481+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21bc005300 con 0x7f21dc098630 2024-01-28T12:56:34.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.481+0000 7f21c3fff640 1 -- 172.21.15.31:0/507152371 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f21cc00fa90 con 0x7f21dc098630 2024-01-28T12:56:34.479 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.483+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f21c8037a90 msgr2=0x7f21c8039fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.483+0000 7f21e1c2d640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f21c8037a90 0x7f21c8039fa0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.483+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc098630 msgr2=0x7f21dc08e450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.483+0000 7f21e1c2d640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc098630 0x7f21dc08e450 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.484+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 shutdown_connections 2024-01-28T12:56:34.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.484+0000 7f21e1c2d640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f21c8037a90 0x7f21c8039fa0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.484+0000 7f21e1c2d640 1 --2- 172.21.15.31:0/507152371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc098630 0x7f21dc08e450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.484+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 >> 172.21.15.31:0/507152371 conn(0x7f21dc09f030 msgr2=0x7f21dc091420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.484+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 shutdown_connections 2024-01-28T12:56:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.484+0000 7f21e1c2d640 1 -- 172.21.15.31:0/507152371 wait complete. 2024-01-28T12:56:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.485+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7e50005300 con 0x7f7e68099fd0 2024-01-28T12:56:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.485+0000 7f7e5e7fc640 1 -- 172.21.15.31:0/2089136352 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f7e6400ea20 con 0x7f7e68099fd0 2024-01-28T12:56:34.484 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7e4c037a40 msgr2=0x7f7e4c039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7e4c037a40 0x7f7e4c039f50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e68099fd0 msgr2=0x7f7e681b54e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e68099fd0 0x7f7e681b54e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 shutdown_connections 2024-01-28T12:56:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7e4c037a40 0x7f7e4c039f50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 --2- 172.21.15.31:0/2089136352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e68099fd0 0x7f7e681b54e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 >> 172.21.15.31:0/2089136352 conn(0x7f7e6809f030 msgr2=0x7f7e6809ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.1 2024-01-28T12:56:34.487 DEBUG:teuthology.parallel:result is None 2024-01-28T12:56:34.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 shutdown_connections 2024-01-28T12:56:34.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.487+0000 7f7e6ff1e640 1 -- 172.21.15.31:0/2089136352 wait complete. 2024-01-28T12:56:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.495+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d1c06c850 con 0x7f4d1c09a560 2024-01-28T12:56:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.495+0000 7f4d037fe640 1 -- 172.21.15.31:0/2767401364 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f4d0c00a120 con 0x7f4d1c09a560 2024-01-28T12:56:34.493 INFO:teuthology.orchestra.run.smithi031.stdout:38654705671 2024-01-28T12:56:34.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.2 2024-01-28T12:56:34.495 DEBUG:teuthology.parallel:result is None 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f4d08037a30 msgr2=0x7f4d08039f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f4d08037a30 0x7f4d08039f40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 msgr2=0x7f4d1c11d860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c11d860 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 shutdown_connections 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f4d08037a30 0x7f4d08039f40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 --2- 172.21.15.31:0/2767401364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c09a560 0x7f4d1c11d860 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 >> 172.21.15.31:0/2767401364 conn(0x7f4d1c09ede0 msgr2=0x7f4d1c12e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 shutdown_connections 2024-01-28T12:56:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.498+0000 7f4d2155e640 1 -- 172.21.15.31:0/2767401364 wait complete. 2024-01-28T12:56:34.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.3 2024-01-28T12:56:34.498 DEBUG:teuthology.parallel:result is None 2024-01-28T12:56:34.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.0 2024-01-28T12:56:34.506 DEBUG:teuthology.parallel:result is None 2024-01-28T12:56:34.506 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-28T12:56:34.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-28T12:56:34.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.650+0000 7f91fb2f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.654+0000 7f91fb2f0640 1 Processor -- start 2024-01-28T12:56:34.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.654+0000 7f91fb2f0640 1 -- start start 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91fb2f0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91f409ab60 0x7f91f4135540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91fb2f0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f406e800 con 0x7f91f409a040 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91fb2f0640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f40986a0 con 0x7f91f409ab60 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91f888b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91f409ab60 0x7f91f4135540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91f908c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f91f409a040 0x7f91f409a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60972/0 (socket says 172.21.15.31:60972) 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91f908c640 1 -- 172.21.15.31:0/776506070 learned_addr learned my addr 172.21.15.31:0/776506070 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91f888b640 1 -- 172.21.15.31:0/776506070 >> v1:172.21.15.31:6789/0 conn(0x7f91f409a040 legacy=0x7f91f409a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91f888b640 1 -- 172.21.15.31:0/776506070 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f406ef20 con 0x7f91f409ab60 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.655+0000 7f91ebfff640 1 -- 172.21.15.31:0/776506070 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037097339 0 0) 0x7f91f406e800 con 0x7f91f409a040 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91f888b640 1 --2- 172.21.15.31:0/776506070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409ab60 0x7f91f4135540 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01e13972cbba6ac server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91ebfff640 1 -- 172.21.15.31:0/776506070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f91e4009ce0 con 0x7f91f409ab60 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91ebfff640 1 -- 172.21.15.31:0/776506070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f91e4009e40 con 0x7f91f409ab60 2024-01-28T12:56:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91ebfff640 1 -- 172.21.15.31:0/776506070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f91e400e360 con 0x7f91f409ab60 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91fb2f0640 1 -- 172.21.15.31:0/776506070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409ab60 msgr2=0x7f91f4135540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91fb2f0640 1 --2- 172.21.15.31:0/776506070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409ab60 0x7f91f4135540 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91fb2f0640 1 -- 172.21.15.31:0/776506070 shutdown_connections 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91fb2f0640 1 --2- 172.21.15.31:0/776506070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409ab60 0x7f91f4135540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91fb2f0640 1 -- 172.21.15.31:0/776506070 >> 172.21.15.31:0/776506070 conn(0x7f91f409e620 msgr2=0x7f91f409bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91fb2f0640 1 -- 172.21.15.31:0/776506070 shutdown_connections 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.656+0000 7f91fb2f0640 1 -- 172.21.15.31:0/776506070 wait complete. 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.657+0000 7f91fb2f0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.658+0000 7f91fb2f0640 1 Processor -- start 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.658+0000 7f91fb2f0640 1 -- start start 2024-01-28T12:56:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.658+0000 7f91fb2f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409a040 0x7f91f41b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.658+0000 7f91fb2f0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f41b8d40 con 0x7f91f409a040 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.658+0000 7f91f908c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409a040 0x7f91f41b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.658+0000 7f91f908c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409a040 0x7f91f41b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57114/0 (socket says 172.21.15.31:57114) 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.658+0000 7f91f908c640 1 -- 172.21.15.31:0/3799115915 learned_addr learned my addr 172.21.15.31:0/3799115915 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.659+0000 7f91f908c640 1 -- 172.21.15.31:0/3799115915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f406bfb0 con 0x7f91f409a040 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.659+0000 7f91f908c640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409a040 0x7f91f41b5520 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.659+0000 7f91e9ffb640 1 -- 172.21.15.31:0/3799115915 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f91ec004af0 con 0x7f91f409a040 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.659+0000 7f91e9ffb640 1 -- 172.21.15.31:0/3799115915 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f91ec004c50 con 0x7f91f409a040 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.659+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f41b75a0 con 0x7f91f409a040 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.659+0000 7f91e9ffb640 1 -- 172.21.15.31:0/3799115915 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f91ec007120 con 0x7f91f409a040 2024-01-28T12:56:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.659+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f41b8a70 con 0x7f91f409a040 2024-01-28T12:56:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.660+0000 7f91e9ffb640 1 -- 172.21.15.31:0/3799115915 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f91ec0072d0 con 0x7f91f409a040 2024-01-28T12:56:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.660+0000 7f91e9ffb640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f91d8037a40 0x7f91d8039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.660+0000 7f91e9ffb640 1 -- 172.21.15.31:0/3799115915 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f91ec046de0 con 0x7f91f409a040 2024-01-28T12:56:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.660+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f406bfb0 con 0x7f91f409a040 2024-01-28T12:56:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.660+0000 7f91f888b640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f91d8037a40 0x7f91d8039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.661+0000 7f91f888b640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f91d8037a40 0x7f91d8039f50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.665+0000 7f91e9ffb640 1 -- 172.21.15.31:0/3799115915 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f91f406bfb0 con 0x7f91f409a040 2024-01-28T12:56:34.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.841+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 --> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91f40a0c20 con 0x7f91d8037a40 2024-01-28T12:56:34.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.843+0000 7f91e9ffb640 1 -- 172.21.15.31:0/3799115915 <== mgr.4099 v2:172.21.15.137:6800/1465424679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+30051 (crc 0 0 0) 0x7f91f40a0c20 con 0x7f91d8037a40 2024-01-28T12:56:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:56:34.841 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f91d8037a40 msgr2=0x7f91d8039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f91d8037a40 0x7f91d8039f50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409a040 msgr2=0x7f91f41b5520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409a040 0x7f91f41b5520 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 shutdown_connections 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f91d8037a40 0x7f91d8039f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 --2- 172.21.15.31:0/3799115915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f409a040 0x7f91f41b5520 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 >> 172.21.15.31:0/3799115915 conn(0x7f91f409e620 msgr2=0x7f91f409f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 shutdown_connections 2024-01-28T12:56:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.845+0000 7f91fb2f0640 1 -- 172.21.15.31:0/3799115915 wait complete. 2024-01-28T12:56:34.855 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-01-28T12:56:34.083002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109808,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320592,"statfs":{"total":429496729600,"available":429384286208,"internally_reserved":0,"allocated":1662976,"data_stored":1299744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.978949+0000","last_change":"2024-01-28T12:56:32.979330+0000","last_active":"2024-01-28T12:56:32.978949+0000","last_peered":"2024-01-28T12:56:32.978949+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.976458+0000","last_became_peered":"2024-01-28T12:56:32.976458+0000","last_unstale":"2024-01-28T12:56:32.978949+0000","last_undegraded":"2024-01-28T12:56:32.978949+0000","last_fullsized":"2024-01-28T12:56:32.978949+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-28T12:56:33.932975+0000","last_change":"2024-01-28T12:56:32.993162+0000","last_active":"2024-01-28T12:56:33.932975+0000","last_peered":"2024-01-28T12:56:33.932975+0000","last_clean":"2024-01-28T12:56:33.932975+0000","last_became_active":"2024-01-28T12:56:32.988926+0000","last_became_peered":"2024-01-28T12:56:32.988926+0000","last_unstale":"2024-01-28T12:56:33.932975+0000","last_undegraded":"2024-01-28T12:56:33.932975+0000","last_fullsized":"2024-01-28T12:56:33.932975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.984955+0000","last_change":"2024-01-28T12:56:32.985340+0000","last_active":"2024-01-28T12:56:32.984955+0000","last_peered":"2024-01-28T12:56:32.984955+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.982359+0000","last_became_peered":"2024-01-28T12:56:32.982359+0000","last_unstale":"2024-01-28T12:56:32.984955+0000","last_undegraded":"2024-01-28T12:56:32.984955+0000","last_fullsized":"2024-01-28T12:56:32.984955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-28T12:56:33.933775+0000","last_change":"2024-01-28T12:56:32.982831+0000","last_active":"2024-01-28T12:56:33.933775+0000","last_peered":"2024-01-28T12:56:33.933775+0000","last_clean":"2024-01-28T12:56:33.933775+0000","last_became_active":"2024-01-28T12:56:32.974936+0000","last_became_peered":"2024-01-28T12:56:32.974936+0000","last_unstale":"2024-01-28T12:56:33.933775+0000","last_undegraded":"2024-01-28T12:56:33.933775+0000","last_fullsized":"2024-01-28T12:56:33.933775+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.984361+0000","last_change":"2024-01-28T12:56:32.984716+0000","last_active":"2024-01-28T12:56:32.984361+0000","last_peered":"2024-01-28T12:56:32.984361+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.981684+0000","last_became_peered":"2024-01-28T12:56:32.981684+0000","last_unstale":"2024-01-28T12:56:32.984361+0000","last_undegraded":"2024-01-28T12:56:32.984361+0000","last_fullsized":"2024-01-28T12:56:32.984361+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.984018+0000","last_change":"2024-01-28T12:56:32.984559+0000","last_active":"2024-01-28T12:56:32.984018+0000","last_peered":"2024-01-28T12:56:32.984018+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.981136+0000","last_became_peered":"2024-01-28T12:56:32.981136+0000","last_unstale":"2024-01-28T12:56:32.984018+0000","last_undegraded":"2024-01-28T12:56:32.984018+0000","last_fullsized":"2024-01-28T12:56:32.984018+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.983372+0000","last_change":"2024-01-28T12:56:32.983756+0000","last_active":"2024-01-28T12:56:32.983372+0000","last_peered":"2024-01-28T12:56:32.983372+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.980691+0000","last_became_peered":"2024-01-28T12:56:32.980691+0000","last_unstale":"2024-01-28T12:56:32.983372+0000","last_undegraded":"2024-01-28T12:56:32.983372+0000","last_fullsized":"2024-01-28T12:56:32.983372+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.990472+0000","last_change":"2024-01-28T12:56:32.990847+0000","last_active":"2024-01-28T12:56:32.990472+0000","last_peered":"2024-01-28T12:56:32.990472+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.987867+0000","last_became_peered":"2024-01-28T12:56:32.987867+0000","last_unstale":"2024-01-28T12:56:32.990472+0000","last_undegraded":"2024-01-28T12:56:32.990472+0000","last_fullsized":"2024-01-28T12:56:32.990472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.935653+0000","last_change":"2024-01-28T12:56:29.935903+0000","last_active":"2024-01-28T12:56:32.935653+0000","last_peered":"2024-01-28T12:56:32.935653+0000","last_clean":"2024-01-28T12:56:32.935653+0000","last_became_active":"2024-01-28T12:56:29.932292+0000","last_became_peered":"2024-01-28T12:56:29.932292+0000","last_unstale":"2024-01-28T12:56:32.935653+0000","last_undegraded":"2024-01-28T12:56:32.935653+0000","last_fullsized":"2024-01-28T12:56:32.935653+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:28.894427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:28.894427+0000","last_clean_scrub_stamp":"2024-01-28T12:56:28.894427+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2024-01-28T12:56:34.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-28T12:56:34.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.961+0000 7f87f3915640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.964+0000 7f87f3915640 1 Processor -- start 2024-01-28T12:56:34.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.965+0000 7f87f3915640 1 -- start start 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.965+0000 7f87f3915640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87ec09adb0 0x7f87ec09b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.965+0000 7f87f3915640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec09a420 con 0x7f87ec133220 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.965+0000 7f87f3915640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec09e4d0 con 0x7f87ec09adb0 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.965+0000 7f87f16b1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87ec09adb0 0x7f87ec09b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.965+0000 7f87f0eb0640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f87ec133220 0x7f87ec133630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60982/0 (socket says 172.21.15.31:60982) 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.965+0000 7f87f0eb0640 1 -- 172.21.15.31:0/3217622795 learned_addr learned my addr 172.21.15.31:0/3217622795 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.966+0000 7f87f16b1640 1 -- 172.21.15.31:0/3217622795 >> v1:172.21.15.31:6789/0 conn(0x7f87ec133220 legacy=0x7f87ec133630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.966+0000 7f87f16b1640 1 -- 172.21.15.31:0/3217622795 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec09a960 con 0x7f87ec09adb0 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.966+0000 7f87e3fff640 1 -- 172.21.15.31:0/3217622795 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875204480 0 0) 0x7f87ec09a420 con 0x7f87ec133220 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.966+0000 7f87f16b1640 1 --2- 172.21.15.31:0/3217622795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec09b1c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82255ca0bd69894 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.966+0000 7f87e3fff640 1 -- 172.21.15.31:0/3217622795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f87e8009ce0 con 0x7f87ec09adb0 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.966+0000 7f87e3fff640 1 -- 172.21.15.31:0/3217622795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f87e8009e40 con 0x7f87ec09adb0 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.966+0000 7f87e3fff640 1 -- 172.21.15.31:0/3217622795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f87e800e360 con 0x7f87ec09adb0 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 1 -- 172.21.15.31:0/3217622795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 msgr2=0x7f87ec09b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 1 --2- 172.21.15.31:0/3217622795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec09b1c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 1 -- 172.21.15.31:0/3217622795 shutdown_connections 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 1 --2- 172.21.15.31:0/3217622795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec09b1c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 1 -- 172.21.15.31:0/3217622795 >> 172.21.15.31:0/3217622795 conn(0x7f87ec0a6060 msgr2=0x7f87ec0989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 1 -- 172.21.15.31:0/3217622795 shutdown_connections 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 1 -- 172.21.15.31:0/3217622795 wait complete. 2024-01-28T12:56:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.967+0000 7f87f3915640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.968+0000 7f87f3915640 1 Processor -- start 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.968+0000 7f87f3915640 1 -- start start 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f3915640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec1c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f3915640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1c45d0 con 0x7f87ec09adb0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f16b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec1c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f16b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec1c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57134/0 (socket says 172.21.15.31:57134) 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f16b1640 1 -- 172.21.15.31:0/2240493338 learned_addr learned my addr 172.21.15.31:0/2240493338 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f16b1640 1 -- 172.21.15.31:0/2240493338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec1c4800 con 0x7f87ec09adb0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f16b1640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec1c4ba0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87e1ffb640 1 -- 172.21.15.31:0/2240493338 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f87e8014410 con 0x7f87ec09adb0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87e1ffb640 1 -- 172.21.15.31:0/2240493338 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f87e8014570 con 0x7f87ec09adb0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec120750 con 0x7f87ec09adb0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.969+0000 7f87e1ffb640 1 -- 172.21.15.31:0/2240493338 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f87e8014810 con 0x7f87ec09adb0 2024-01-28T12:56:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.970+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec11f440 con 0x7f87ec09adb0 2024-01-28T12:56:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.970+0000 7f87e1ffb640 1 -- 172.21.15.31:0/2240493338 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f87e80149c0 con 0x7f87ec09adb0 2024-01-28T12:56:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.970+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec075440 con 0x7f87ec09adb0 2024-01-28T12:56:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.971+0000 7f87e1ffb640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f87d0037a40 0x7f87d0039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.971+0000 7f87e1ffb640 1 -- 172.21.15.31:0/2240493338 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f87e8019040 con 0x7f87ec09adb0 2024-01-28T12:56:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.971+0000 7f87f0eb0640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f87d0037a40 0x7f87d0039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.971+0000 7f87f0eb0640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f87d0037a40 0x7f87d0039f50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:34.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:34.975+0000 7f87e1ffb640 1 -- 172.21.15.31:0/2240493338 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f87e800f710 con 0x7f87ec09adb0 2024-01-28T12:56:35.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.150+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 --> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87ec09d8c0 con 0x7f87d0037a40 2024-01-28T12:56:35.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.152+0000 7f87e1ffb640 1 -- 172.21.15.31:0/2240493338 <== mgr.4099 v2:172.21.15.137:6800/1465424679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+30051 (crc 0 0 0) 0x7f87dc001250 con 0x7f87d0037a40 2024-01-28T12:56:35.150 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:56:35.150 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-01-28T12:56:35.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f87d0037a40 msgr2=0x7f87d0039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:35.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f87d0037a40 0x7f87d0039f50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 msgr2=0x7f87ec1c4ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:35.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec1c4ba0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 shutdown_connections 2024-01-28T12:56:35.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f87d0037a40 0x7f87d0039f50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 --2- 172.21.15.31:0/2240493338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec09adb0 0x7f87ec1c4ba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 >> 172.21.15.31:0/2240493338 conn(0x7f87ec0a6060 msgr2=0x7f87ec0a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:35.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 shutdown_connections 2024-01-28T12:56:35.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.154+0000 7f87f3915640 1 -- 172.21.15.31:0/2240493338 wait complete. 2024-01-28T12:56:35.162 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-01-28T12:56:34.846961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109808,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320592,"statfs":{"total":429496729600,"available":429384286208,"internally_reserved":0,"allocated":1662976,"data_stored":1299744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.763959"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.978949+0000","last_change":"2024-01-28T12:56:32.979330+0000","last_active":"2024-01-28T12:56:32.978949+0000","last_peered":"2024-01-28T12:56:32.978949+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.976458+0000","last_became_peered":"2024-01-28T12:56:32.976458+0000","last_unstale":"2024-01-28T12:56:32.978949+0000","last_undegraded":"2024-01-28T12:56:32.978949+0000","last_fullsized":"2024-01-28T12:56:32.978949+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-28T12:56:33.932975+0000","last_change":"2024-01-28T12:56:32.993162+0000","last_active":"2024-01-28T12:56:33.932975+0000","last_peered":"2024-01-28T12:56:33.932975+0000","last_clean":"2024-01-28T12:56:33.932975+0000","last_became_active":"2024-01-28T12:56:32.988926+0000","last_became_peered":"2024-01-28T12:56:32.988926+0000","last_unstale":"2024-01-28T12:56:33.932975+0000","last_undegraded":"2024-01-28T12:56:33.932975+0000","last_fullsized":"2024-01-28T12:56:33.932975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.984955+0000","last_change":"2024-01-28T12:56:32.985340+0000","last_active":"2024-01-28T12:56:32.984955+0000","last_peered":"2024-01-28T12:56:32.984955+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.982359+0000","last_became_peered":"2024-01-28T12:56:32.982359+0000","last_unstale":"2024-01-28T12:56:32.984955+0000","last_undegraded":"2024-01-28T12:56:32.984955+0000","last_fullsized":"2024-01-28T12:56:32.984955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-28T12:56:33.933775+0000","last_change":"2024-01-28T12:56:32.982831+0000","last_active":"2024-01-28T12:56:33.933775+0000","last_peered":"2024-01-28T12:56:33.933775+0000","last_clean":"2024-01-28T12:56:33.933775+0000","last_became_active":"2024-01-28T12:56:32.974936+0000","last_became_peered":"2024-01-28T12:56:32.974936+0000","last_unstale":"2024-01-28T12:56:33.933775+0000","last_undegraded":"2024-01-28T12:56:33.933775+0000","last_fullsized":"2024-01-28T12:56:33.933775+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.984361+0000","last_change":"2024-01-28T12:56:32.984716+0000","last_active":"2024-01-28T12:56:32.984361+0000","last_peered":"2024-01-28T12:56:32.984361+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.981684+0000","last_became_peered":"2024-01-28T12:56:32.981684+0000","last_unstale":"2024-01-28T12:56:32.984361+0000","last_undegraded":"2024-01-28T12:56:32.984361+0000","last_fullsized":"2024-01-28T12:56:32.984361+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.984018+0000","last_change":"2024-01-28T12:56:32.984559+0000","last_active":"2024-01-28T12:56:32.984018+0000","last_peered":"2024-01-28T12:56:32.984018+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.981136+0000","last_became_peered":"2024-01-28T12:56:32.981136+0000","last_unstale":"2024-01-28T12:56:32.984018+0000","last_undegraded":"2024-01-28T12:56:32.984018+0000","last_fullsized":"2024-01-28T12:56:32.984018+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.983372+0000","last_change":"2024-01-28T12:56:32.983756+0000","last_active":"2024-01-28T12:56:32.983372+0000","last_peered":"2024-01-28T12:56:32.983372+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.980691+0000","last_became_peered":"2024-01-28T12:56:32.980691+0000","last_unstale":"2024-01-28T12:56:32.983372+0000","last_undegraded":"2024-01-28T12:56:32.983372+0000","last_fullsized":"2024-01-28T12:56:32.983372+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.990472+0000","last_change":"2024-01-28T12:56:32.990847+0000","last_active":"2024-01-28T12:56:32.990472+0000","last_peered":"2024-01-28T12:56:32.990472+0000","last_clean":"2024-01-28T12:56:31.906365+0000","last_became_active":"2024-01-28T12:56:32.987867+0000","last_became_peered":"2024-01-28T12:56:32.987867+0000","last_unstale":"2024-01-28T12:56:32.990472+0000","last_undegraded":"2024-01-28T12:56:32.990472+0000","last_fullsized":"2024-01-28T12:56:32.990472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:31.906365+0000","last_clean_scrub_stamp":"2024-01-28T12:56:31.906365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-01-28T12:56:32.935653+0000","last_change":"2024-01-28T12:56:29.935903+0000","last_active":"2024-01-28T12:56:32.935653+0000","last_peered":"2024-01-28T12:56:32.935653+0000","last_clean":"2024-01-28T12:56:32.935653+0000","last_became_active":"2024-01-28T12:56:29.932292+0000","last_became_peered":"2024-01-28T12:56:29.932292+0000","last_unstale":"2024-01-28T12:56:32.935653+0000","last_undegraded":"2024-01-28T12:56:32.935653+0000","last_fullsized":"2024-01-28T12:56:32.935653+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-28T12:56:28.894427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-28T12:56:28.894427+0000","last_clean_scrub_stamp":"2024-01-28T12:56:28.894427+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2024-01-28T12:56:35.163 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-28T12:56:35.163 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-28T12:56:35.163 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-28T12:56:35.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-28T12:56:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.268+0000 7f7c1f75e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.271+0000 7f7c1f75e640 1 Processor -- start 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.271+0000 7f7c1f75e640 1 -- start start 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1f75e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c18097b70 0x7f7c18136760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1f75e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c18070ee0 con 0x7f7c18097050 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1f75e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c180727a0 con 0x7f7c18097b70 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1ccf9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c18097b70 0x7f7c18136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1d4fa640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7c18097050 0x7f7c18097460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60994/0 (socket says 172.21.15.31:60994) 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1d4fa640 1 -- 172.21.15.31:0/3665073327 learned_addr learned my addr 172.21.15.31:0/3665073327 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1ccf9640 1 -- 172.21.15.31:0/3665073327 >> v1:172.21.15.31:6789/0 conn(0x7f7c18097050 legacy=0x7f7c18097460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1ccf9640 1 -- 172.21.15.31:0/3665073327 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c18098270 con 0x7f7c18097b70 2024-01-28T12:56:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.272+0000 7f7c1ccf9640 1 --2- 172.21.15.31:0/3665073327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097b70 0x7f7c18136760 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b1f76eb510b4aa server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c17fff640 1 -- 172.21.15.31:0/3665073327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7c0c009ce0 con 0x7f7c18097b70 2024-01-28T12:56:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c17fff640 1 -- 172.21.15.31:0/3665073327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7c0c009e40 con 0x7f7c18097b70 2024-01-28T12:56:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c17fff640 1 -- 172.21.15.31:0/3665073327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7c0c00e360 con 0x7f7c18097b70 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c1f75e640 1 -- 172.21.15.31:0/3665073327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097b70 msgr2=0x7f7c18136760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c1f75e640 1 --2- 172.21.15.31:0/3665073327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097b70 0x7f7c18136760 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c1f75e640 1 -- 172.21.15.31:0/3665073327 shutdown_connections 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c1f75e640 1 --2- 172.21.15.31:0/3665073327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097b70 0x7f7c18136760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c1f75e640 1 -- 172.21.15.31:0/3665073327 >> 172.21.15.31:0/3665073327 conn(0x7f7c1809f030 msgr2=0x7f7c1809f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c1f75e640 1 -- 172.21.15.31:0/3665073327 shutdown_connections 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.273+0000 7f7c1f75e640 1 -- 172.21.15.31:0/3665073327 wait complete. 2024-01-28T12:56:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.274+0000 7f7c1f75e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.274+0000 7f7c1f75e640 1 Processor -- start 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.274+0000 7f7c1f75e640 1 -- start start 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.275+0000 7f7c1f75e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097050 0x7f7c1812b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.275+0000 7f7c1f75e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1812e180 con 0x7f7c18097050 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.275+0000 7f7c1d4fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097050 0x7f7c1812b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.275+0000 7f7c1d4fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097050 0x7f7c1812b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57146/0 (socket says 172.21.15.31:57146) 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.275+0000 7f7c1d4fa640 1 -- 172.21.15.31:0/2300845763 learned_addr learned my addr 172.21.15.31:0/2300845763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.275+0000 7f7c1d4fa640 1 -- 172.21.15.31:0/2300845763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c18072cb0 con 0x7f7c18097050 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.276+0000 7f7c1d4fa640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097050 0x7f7c1812b100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.276+0000 7f7c15ffb640 1 -- 172.21.15.31:0/2300845763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7c100059d0 con 0x7f7c18097050 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.276+0000 7f7c15ffb640 1 -- 172.21.15.31:0/2300845763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7c10005b30 con 0x7f7c18097050 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.276+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1812f0b0 con 0x7f7c18097050 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.276+0000 7f7c15ffb640 1 -- 172.21.15.31:0/2300845763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7c10005dd0 con 0x7f7c18097050 2024-01-28T12:56:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.276+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1806beb0 con 0x7f7c18097050 2024-01-28T12:56:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.276+0000 7f7c15ffb640 1 -- 172.21.15.31:0/2300845763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f7c100070b0 con 0x7f7c18097050 2024-01-28T12:56:35.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.277+0000 7f7c15ffb640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7c04037a40 0x7f7c04039f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T12:56:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.277+0000 7f7c15ffb640 1 -- 172.21.15.31:0/2300845763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7c10046e40 con 0x7f7c18097050 2024-01-28T12:56:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.277+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c18075440 con 0x7f7c18097050 2024-01-28T12:56:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.277+0000 7f7c1ccf9640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7c04037a40 0x7f7c04039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T12:56:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.277+0000 7f7c1ccf9640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7c04037a40 0x7f7c04039f50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T12:56:35.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.282+0000 7f7c15ffb640 1 -- 172.21.15.31:0/2300845763 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (crc 0 0 0) 0x7f7c1000d330 con 0x7f7c18097050 2024-01-28T12:56:35.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.496+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7c1812d4a0 con 0x7f7c18097050 2024-01-28T12:56:35.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.497+0000 7f7c15ffb640 1 -- 172.21.15.31:0/2300845763 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (crc 0 0 0) 0x7f7c1000d150 con 0x7f7c18097050 2024-01-28T12:56:35.495 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-01-28T12:56:35.496 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-28T12:56:35.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7c04037a40 msgr2=0x7f7c04039f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:35.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7c04037a40 0x7f7c04039f50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097050 msgr2=0x7f7c1812b100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T12:56:35.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097050 0x7f7c1812b100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 shutdown_connections 2024-01-28T12:56:35.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f7c04037a40 0x7f7c04039f50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 --2- 172.21.15.31:0/2300845763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18097050 0x7f7c1812b100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T12:56:35.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 >> 172.21.15.31:0/2300845763 conn(0x7f7c1809f030 msgr2=0x7f7c181b91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T12:56:35.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 shutdown_connections 2024-01-28T12:56:35.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T12:56:35.499+0000 7f7c1f75e640 1 -- 172.21.15.31:0/2300845763 wait complete. 2024-01-28T12:56:35.507 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-28T12:56:35.508 INFO:teuthology.run_tasks:Running task workunit... 2024-01-28T12:56:35.521 INFO:tasks.workunit:Pulling workunits from ref 7eb9e33f53ab64f23bcf0a2a31509f3442110e4e 2024-01-28T12:56:35.521 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-01-28T12:56:35.521 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-01-28T12:56:35.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T12:56:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-01-28T12:56:35.536 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-01-28T12:56:35.589 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-01-28T12:56:35.590 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-01-28T12:56:35.644 INFO:tasks.workunit:timeout=3h 2024-01-28T12:56:35.644 INFO:tasks.workunit:cleanup=True 2024-01-28T12:56:35.644 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7eb9e33f53ab64f23bcf0a2a31509f3442110e4e 2024-01-28T12:56:35.698 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to '7eb9e33f53ab64f23bcf0a2a31509f3442110e4e'. 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-01-28T12:57:26.043 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-01-28T12:57:26.044 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at 7eb9e33f53a Merge pull request #55281 from Matan-B/wip-matanb-crimson-cyanstore-rmcoll 2024-01-28T12:57:26.045 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-01-28T12:57:26.107 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-01-28T12:57:26.110 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-28T12:57:26.110 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-01-28T12:57:26.845 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-01-28T12:57:26.920 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-01-28T12:57:26.962 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-28T12:57:26.963 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-28T12:57:26.963 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-01-28T12:57:27.002 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-28T12:57:27.005 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-01-28T12:57:27.005 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-01-28T12:57:27.059 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-01-28T12:57:27.060 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-01-28T12:57:27.061 DEBUG:teuthology.orchestra.run.smithi031:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7eb9e33f53ab64f23bcf0a2a31509f3442110e4e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-01-28T12:57:27.118 INFO:tasks.workunit.client.0.smithi031.stderr:+ parallel=1 2024-01-28T12:57:27.118 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' --crimson = --serial ']' 2024-01-28T12:57:27.118 INFO:tasks.workunit.client.0.smithi031.stderr:+ crimson=0 2024-01-28T12:57:27.118 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' --crimson = --crimson ']' 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ parallel=0 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ crimson=1 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ color= 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' -t 1 ']' 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ GTEST_OUTPUT=xml:/home/ubuntu/cephtest/archive/gtest_xml_report 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ declare -A pids 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T12:57:27.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_aio 2024-01-28T12:57:27.133 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T12:57:27.133 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 42 tests from 2 test suites. 2024-01-28T12:57:27.133 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T12:57:27.133 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 26 tests from LibRadosAio 2024-01-28T12:57:27.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.TooBig 2024-01-28T12:57:29.691 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.TooBig (2558 ms) 2024-01-28T12:57:29.691 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-01-28T12:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.SimpleWrite (3008 ms) 2024-01-28T12:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-01-28T12:57:35.719 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.WaitForSafe (3020 ms) 2024-01-28T12:57:35.719 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-01-28T12:57:38.736 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTrip (3017 ms) 2024-01-28T12:57:38.737 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-01-28T12:57:41.749 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTrip2 (3013 ms) 2024-01-28T12:57:41.749 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-01-28T12:57:44.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTrip3 (3011 ms) 2024-01-28T12:57:44.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-01-28T12:57:47.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripAppend (3023 ms) 2024-01-28T12:57:47.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-01-28T12:57:50.795 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RemoveTest (3012 ms) 2024-01-28T12:57:50.795 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-01-28T12:57:53.822 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3027 ms) 2024-01-28T12:57:53.822 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RmXattr 2024-01-28T12:57:56.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RmXattr (3008 ms) 2024-01-28T12:57:56.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.XattrIter 2024-01-28T12:57:59.845 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.XattrIter (3015 ms) 2024-01-28T12:57:59.845 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.IsComplete 2024-01-28T12:58:02.860 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.IsComplete (3016 ms) 2024-01-28T12:58:02.861 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.IsSafe 2024-01-28T12:58:05.880 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.IsSafe (3019 ms) 2024-01-28T12:58:05.881 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-01-28T12:58:08.889 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.ReturnValue (3009 ms) 2024-01-28T12:58:08.889 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.Flush 2024-01-28T12:58:11.911 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.Flush (3023 ms) 2024-01-28T12:58:11.912 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-01-28T12:58:14.925 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.FlushAsync (3013 ms) 2024-01-28T12:58:14.925 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-01-28T12:58:17.932 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3008 ms) 2024-01-28T12:58:17.933 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-01-28T12:58:20.654 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (2721 ms) 2024-01-28T12:58:20.654 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-01-28T12:58:23.666 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.SimpleStat (3012 ms) 2024-01-28T12:58:23.666 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-01-28T12:58:26.684 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.OperateMtime (3017 ms) 2024-01-28T12:58:26.684 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-01-28T12:58:29.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.Operate2Mtime (3014 ms) 2024-01-28T12:58:29.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-01-28T12:58:32.707 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.SimpleStatNS (3010 ms) 2024-01-28T12:58:32.707 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.StatRemove 2024-01-28T12:58:35.657 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.StatRemove (2950 ms) 2024-01-28T12:58:35.657 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-01-28T12:58:38.668 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.ExecuteClass (3012 ms) 2024-01-28T12:58:38.669 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-01-28T12:58:41.679 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.MultiWrite (3010 ms) 2024-01-28T12:58:41.679 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-01-28T12:58:44.692 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.AioUnlock (3012 ms) 2024-01-28T12:58:44.692 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 26 tests from LibRadosAio (77558 ms total) 2024-01-28T12:58:44.692 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 16 tests from LibRadosAioEC 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1100: Skipped 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1132: Skipped 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1149: Skipped 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-01-28T12:58:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1182: Skipped 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1215: Skipped 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1286: Skipped 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1326: Skipped 2024-01-28T12:58:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.695 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-01-28T12:58:44.695 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-01-28T12:58:44.695 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1366: Skipped 2024-01-28T12:58:44.695 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.695 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-01-28T12:58:44.695 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.Flush 2024-01-28T12:58:44.695 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1385: Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1415: Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1454: Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1498: Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-01-28T12:58:44.696 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1532: Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1593: Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1649: Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio.cc:1681: Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T12:58:44.697 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (0 ms) 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 16 tests from LibRadosAioEC (0 ms total) 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 42 tests from 2 test suites ran. (77558 ms total) 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 26 tests. 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 16 tests, listed below: 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-01-28T12:58:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-01-28T12:58:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-01-28T12:58:44.700 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T12:58:44.700 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T12:58:44.700 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_aio_pp 2024-01-28T12:58:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T12:58:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 57 tests from 4 test suites. 2024-01-28T12:58:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T12:58:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 32 tests from LibRadosAio 2024-01-28T12:58:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-01-28T12:58:47.666 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.TooBigPP (2952 ms) 2024-01-28T12:58:47.666 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-01-28T12:59:08.672 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.PoolQuotaPP (21007 ms) 2024-01-28T12:59:08.672 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-01-28T12:59:14.692 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.SimpleWritePP (6020 ms) 2024-01-28T12:59:14.692 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-01-28T12:59:17.703 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.WaitForSafePP (3011 ms) 2024-01-28T12:59:17.703 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-01-28T12:59:20.666 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripPP (2963 ms) 2024-01-28T12:59:20.666 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-01-28T12:59:23.681 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3015 ms) 2024-01-28T12:59:23.682 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-01-28T12:59:26.689 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3007 ms) 2024-01-28T12:59:26.690 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-01-28T12:59:29.702 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3013 ms) 2024-01-28T12:59:29.702 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-01-28T12:59:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.IsCompletePP (3001 ms) 2024-01-28T12:59:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-01-28T12:59:36.718 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.IsSafePP (4015 ms) 2024-01-28T12:59:36.718 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-01-28T12:59:39.741 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.ReturnValuePP (3023 ms) 2024-01-28T12:59:39.741 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.FlushPP 2024-01-28T12:59:43.685 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.FlushPP (3944 ms) 2024-01-28T12:59:43.685 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-01-28T12:59:46.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3009 ms) 2024-01-28T12:59:46.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-01-28T12:59:50.675 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3980 ms) 2024-01-28T12:59:50.675 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-01-28T12:59:53.687 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (3013 ms) 2024-01-28T12:59:53.687 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-01-28T12:59:58.703 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (5016 ms) 2024-01-28T12:59:58.703 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-01-28T13:00:03.721 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (5018 ms) 2024-01-28T13:00:03.722 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-01-28T13:00:06.737 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3016 ms) 2024-01-28T13:00:06.737 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-01-28T13:00:09.752 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.SimpleStatPP (3015 ms) 2024-01-28T13:00:09.752 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-01-28T13:00:14.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.OperateMtime (5010 ms) 2024-01-28T13:00:14.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-01-28T13:00:17.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.OperateMtime2 (3014 ms) 2024-01-28T13:00:17.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-01-28T13:00:22.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.StatRemovePP (5014 ms) 2024-01-28T13:00:22.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-01-28T13:00:32.836 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.ExecuteClassPP (10046 ms) 2024-01-28T13:00:32.836 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.OmapPP 2024-01-28T13:00:37.863 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.OmapPP (5027 ms) 2024-01-28T13:00:37.863 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-01-28T13:00:48.917 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.MultiWritePP (11054 ms) 2024-01-28T13:00:48.918 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-01-28T13:00:53.949 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.AioUnlockPP (5033 ms) 2024-01-28T13:00:53.950 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-01-28T13:00:59.967 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (6018 ms) 2024-01-28T13:00:59.968 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-01-28T13:01:11.720 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RacingRemovePP (11751 ms) 2024-01-28T13:01:11.720 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-01-28T13:01:20.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (9041 ms) 2024-01-28T13:01:20.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-01-28T13:01:29.805 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (9045 ms) 2024-01-28T13:01:29.806 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-01-28T13:01:32.304 INFO:tasks.workunit.client.0.smithi031.stdout:setting pool EIO 2024-01-28T13:01:41.939 INFO:tasks.workunit.client.0.smithi031.stdout:waiting for 1000 2024-01-28T13:01:42.938 INFO:tasks.workunit.client.0.smithi031.stdout:waiting for 1000 2024-01-28T13:01:43.939 INFO:tasks.workunit.client.0.smithi031.stdout:waiting for 1000 2024-01-28T13:01:44.939 INFO:tasks.workunit.client.0.smithi031.stdout:max_success 0, min_failed 1 2024-01-28T13:01:45.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAio.PoolEIOFlag (15888 ms) 2024-01-28T13:01:45.693 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAio.MultiReads 2024-01-28T13:01:45.693 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:2427: Skipped 2024-01-28T13:01:45.694 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:01:45.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-01-28T13:01:45.694 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 32 tests from LibRadosAio (180980 ms total) 2024-01-28T13:01:45.694 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:01:45.694 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosAioPP 2024-01-28T13:01:45.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-01-28T13:01:50.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (5001 ms) 2024-01-28T13:01:50.694 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-01-28T13:02:16.703 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (26008 ms) 2024-01-28T13:02:16.704 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-01-28T13:02:50.707 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAioPP.RmXattrPP (34005 ms) 2024-01-28T13:02:50.708 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-01-28T13:03:08.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (18073 ms) 2024-01-28T13:03:08.780 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosAioPP (83087 ms total) 2024-01-28T13:03:08.780 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:08.781 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosIoPP 2024-01-28T13:03:08.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-01-28T13:03:19.976 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.XattrListPP (11195 ms) 2024-01-28T13:03:19.976 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosIoPP (11195 ms total) 2024-01-28T13:03:19.976 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 20 tests from LibRadosAioEC 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1384: Skipped 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1419: Skipped 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1436: Skipped 2024-01-28T13:03:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1467: Skipped 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1500: Skipped 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1605: Skipped 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-01-28T13:03:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1637: Skipped 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1705: Skipped 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1742: Skipped 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-01-28T13:03:19.979 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1780: Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1796: Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1824: Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (0 ms) 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1859: Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-01-28T13:03:19.980 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1904: Skipped 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1946: Skipped 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:1977: Skipped 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:2008: Skipped 2024-01-28T13:03:19.981 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:2059: Skipped 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:2089: Skipped 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/aio_cxx.cc:2124: Skipped 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-01-28T13:03:19.982 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 20 tests from LibRadosAioEC (0 ms total) 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 57 tests from 4 test suites ran. (275262 ms total) 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 36 tests. 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 21 tests, listed below: 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-01-28T13:03:19.983 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-01-28T13:03:19.984 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-01-28T13:03:19.985 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-01-28T13:03:19.985 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:03:19.985 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:03:19.985 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_io 2024-01-28T13:03:19.993 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:03:19.993 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 24 tests from 2 test suites. 2024-01-28T13:03:19.993 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:03:19.993 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 14 tests from LibRadosIo 2024-01-28T13:03:21.982 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-01-28T13:03:26.559 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.SimpleWrite (4577 ms) 2024-01-28T13:03:26.559 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.TooBig 2024-01-28T13:03:26.560 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.TooBig (0 ms) 2024-01-28T13:03:26.560 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-01-28T13:03:31.881 INFO:tasks.workunit.client.0.smithi031.stdout:no timeout :/ 2024-01-28T13:03:31.881 INFO:tasks.workunit.client.0.smithi031.stdout:no timeout :/ 2024-01-28T13:03:31.881 INFO:tasks.workunit.client.0.smithi031.stdout:no timeout :/ 2024-01-28T13:03:31.881 INFO:tasks.workunit.client.0.smithi031.stdout:no timeout :/ 2024-01-28T13:03:31.882 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.ReadTimeout (5322 ms) 2024-01-28T13:03:31.882 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-01-28T13:03:32.716 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.RoundTrip (835 ms) 2024-01-28T13:03:32.716 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.Checksum 2024-01-28T13:03:32.729 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.Checksum (14 ms) 2024-01-28T13:03:32.730 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-01-28T13:03:32.750 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (21 ms) 2024-01-28T13:03:32.751 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-01-28T13:03:32.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (40 ms) 2024-01-28T13:03:32.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-01-28T13:03:33.855 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.AppendRoundTrip (1064 ms) 2024-01-28T13:03:33.855 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-01-28T13:03:33.867 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.ZeroLenZero (12 ms) 2024-01-28T13:03:33.867 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.TruncTest 2024-01-28T13:03:33.882 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.TruncTest (16 ms) 2024-01-28T13:03:33.882 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-01-28T13:03:33.897 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.RemoveTest (15 ms) 2024-01-28T13:03:33.897 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-01-28T13:03:33.915 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (17 ms) 2024-01-28T13:03:33.915 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.RmXattr 2024-01-28T13:03:33.959 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.RmXattr (44 ms) 2024-01-28T13:03:33.959 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIo.XattrIter 2024-01-28T13:03:33.994 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIo.XattrIter (36 ms) 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 14 tests from LibRadosIo (12013 ms total) 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 10 tests from LibRadosIoEC 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:168: Skipped 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.758 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (0 ms) 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:176: Skipped 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 10 tests from LibRadosIoEC (0 ms total) 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 24 tests from 2 test suites ran. (14764 ms total) 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 14 tests. 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 10 tests, listed below: 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-01-28T13:03:34.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-01-28T13:03:34.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-01-28T13:03:34.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-01-28T13:03:34.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-01-28T13:03:34.764 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:03:34.764 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:03:34.764 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_io_pp 2024-01-28T13:03:34.781 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:03:34.781 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 37 tests from 2 test suites. 2024-01-28T13:03:34.781 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:03:34.781 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 20 tests from LibRadosIoPP 2024-01-28T13:03:34.781 INFO:tasks.workunit.client.0.smithi031.stdout:seed 33399 2024-01-28T13:03:36.716 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-01-28T13:03:36.716 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-01-28T13:03:36.717 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-01-28T13:03:44.805 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (8089 ms) 2024-01-28T13:03:44.805 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-01-28T13:03:48.720 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.ReadOpPP (3915 ms) 2024-01-28T13:03:48.720 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-01-28T13:03:50.604 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (1884 ms) 2024-01-28T13:03:50.604 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-01-28T13:03:50.932 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.RoundTripPP (316 ms) 2024-01-28T13:03:50.932 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-01-28T13:03:52.101 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (1180 ms) 2024-01-28T13:03:52.101 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-01-28T13:03:52.118 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.Checksum (18 ms) 2024-01-28T13:03:52.118 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-01-28T13:03:52.130 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (12 ms) 2024-01-28T13:03:52.131 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-01-28T13:03:52.313 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (182 ms) 2024-01-28T13:03:52.314 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-01-28T13:03:52.335 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (23 ms) 2024-01-28T13:03:52.335 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-01-28T13:03:52.346 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (10 ms) 2024-01-28T13:03:52.346 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-01-28T13:03:52.362 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (17 ms) 2024-01-28T13:03:52.363 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-01-28T13:03:52.380 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.TruncTestPP (18 ms) 2024-01-28T13:03:52.380 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-01-28T13:03:52.399 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (19 ms) 2024-01-28T13:03:52.400 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-01-28T13:03:52.421 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (22 ms) 2024-01-28T13:03:52.421 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-01-28T13:03:52.472 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.RmXattrPP (51 ms) 2024-01-28T13:03:52.472 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-01-28T13:03:52.493 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.XattrListPP (21 ms) 2024-01-28T13:03:52.493 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-01-28T13:03:52.512 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.CmpExtPP (19 ms) 2024-01-28T13:03:52.512 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-01-28T13:03:52.523 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (11 ms) 2024-01-28T13:03:52.523 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-01-28T13:03:52.538 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (15 ms) 2024-01-28T13:03:52.760 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 20 tests from LibRadosIoPP (15822 ms total) 2024-01-28T13:03:52.760 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:52.760 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 17 tests from LibRadosIoECPP 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.761 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-01-28T13:03:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.763 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-01-28T13:03:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-01-28T13:03:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-01-28T13:03:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-01-28T13:03:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (0 ms) 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.768 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 17 tests from LibRadosIoECPP (0 ms total) 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 37 tests from 2 test suites ran. (17978 ms total) 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 20 tests. 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 17 tests, listed below: 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-01-28T13:03:52.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-01-28T13:03:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-01-28T13:03:52.771 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:03:52.771 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:03:52.771 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_asio 2024-01-28T13:03:52.784 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.786+0000 7f3b5825e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:03:52.789 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.791+0000 7f3b5825e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:03:52.789 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.791+0000 7f3b5825e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:03:52.789 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 12 tests from 1 test suite. 2024-01-28T13:03:52.790 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:03:52.790 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 12 tests from AsioRados 2024-01-28T13:03:52.790 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.792+0000 7f3b5825e640 1 Processor -- start 2024-01-28T13:03:52.790 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.792+0000 7f3b5825e640 1 -- start start 2024-01-28T13:03:52.792 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.793+0000 7f3b5825e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5db5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:03:52.792 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.793+0000 7f3b5825e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c5c375c0 con 0x5571c5c9cb50 2024-01-28T13:03:52.792 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.793+0000 7f3b51b9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5db5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T13:03:52.792 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.794+0000 7f3b51b9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5db5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55540/0 (socket says 172.21.15.31:55540) 2024-01-28T13:03:52.792 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.794+0000 7f3b51b9b640 1 -- 172.21.15.31:0/1365740290 learned_addr learned my addr 172.21.15.31:0/1365740290 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T13:03:52.792 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.794+0000 7f3b51b9b640 1 -- 172.21.15.31:0/1365740290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c5bd8180 con 0x5571c5c9cb50 2024-01-28T13:03:52.793 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.794+0000 7f3b51b9b640 1 --2- 172.21.15.31:0/1365740290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5db5a00 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68356b0f753b941e server_cookie=0 in_seq=0 out_seq=0 2024-01-28T13:03:52.793 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.794+0000 7f3b4bfff640 1 -- 172.21.15.31:0/1365740290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b3c02cc00 con 0x5571c5c9cb50 2024-01-28T13:03:52.793 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.794+0000 7f3b4bfff640 1 -- 172.21.15.31:0/1365740290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b3c02cd60 con 0x5571c5c9cb50 2024-01-28T13:03:52.793 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.794+0000 7f3b4bfff640 1 -- 172.21.15.31:0/1365740290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b3c00a170 con 0x5571c5c9cb50 2024-01-28T13:03:52.793 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.795+0000 7f3b5825e640 1 -- 172.21.15.31:0/1365740290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 msgr2=0x5571c5db5a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.795+0000 7f3b5825e640 1 --2- 172.21.15.31:0/1365740290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5db5a00 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.795+0000 7f3b5825e640 1 -- 172.21.15.31:0/1365740290 shutdown_connections 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.795+0000 7f3b5825e640 1 --2- 172.21.15.31:0/1365740290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5db5a00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.795+0000 7f3b5825e640 1 -- 172.21.15.31:0/1365740290 >> 172.21.15.31:0/1365740290 conn(0x5571c5baae00 msgr2=0x5571c5c9ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.795+0000 7f3b5825e640 1 -- 172.21.15.31:0/1365740290 shutdown_connections 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.795+0000 7f3b5825e640 1 -- 172.21.15.31:0/1365740290 wait complete. 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.796+0000 7f3b5825e640 1 Processor -- start 2024-01-28T13:03:52.794 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.796+0000 7f3b5825e640 1 -- start start 2024-01-28T13:03:52.796 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.797+0000 7f3b5825e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5d18b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:03:52.796 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.797+0000 7f3b5825e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c5c32eb0 con 0x5571c5c9cb50 2024-01-28T13:03:52.796 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b51b9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5d18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T13:03:52.796 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b51b9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5d18b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55552/0 (socket says 172.21.15.31:55552) 2024-01-28T13:03:52.796 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b51b9b640 1 -- 172.21.15.31:0/3277292584 learned_addr learned my addr 172.21.15.31:0/3277292584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-01-28T13:03:52.797 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b51b9b640 1 -- 172.21.15.31:0/3277292584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c5bcb330 con 0x5571c5c9cb50 2024-01-28T13:03:52.797 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b51b9b640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5d18b00 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T13:03:52.797 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b3c03a8a0 con 0x5571c5c9cb50 2024-01-28T13:03:52.797 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b3c03aa00 con 0x5571c5c9cb50 2024-01-28T13:03:52.797 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5571c5bcad90 con 0x5571c5c9cb50 2024-01-28T13:03:52.797 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b3c03aca0 con 0x5571c5c9cb50 2024-01-28T13:03:52.797 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.798+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571c5d1be40 con 0x5571c5c9cb50 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.799+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 44917+0+0 (crc 0 0 0) 0x7f3b3c038450 con 0x5571c5c9cb50 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.799+0000 7f3b4affd640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f3b38036310 0x7f3b38038820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.799+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(367..367 src has 1..367) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3b3c031040 con 0x5571c5c9cb50 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.800+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x5571c5c2ba60 con 0x5571c5c9cb50 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.800+0000 7f3b5139a640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f3b38036310 0x7f3b38038820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.800+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=367) v2 ==== 24+0+0 (crc 0 0 0) 0x7f3b3c0358d0 con 0x5571c5c9cb50 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.800+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x5571c5c3cf40 con 0x5571c5c9cb50 2024-01-28T13:03:52.799 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:52.800+0000 7f3b5139a640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f3b38036310 0x7f3b38038820 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T13:03:53.761 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:53.763+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v368) v1 ==== 43+0+0 (crc 0 0 0) 0x7f3b3c035300 con 0x5571c5c9cb50 2024-01-28T13:03:53.761 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:53.763+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=368}) v3 -- 0x7f3b3803ac30 con 0x5571c5c9cb50 2024-01-28T13:03:53.762 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:53.763+0000 7f3b4affd640 1 -- 172.21.15.31:0/3277292584 <== mon.0 v2:172.21.15.31:3300/0 8 ==== osd_map(368..368 src has 1..368) v4 ==== 651+0+0 (crc 0 0 0) 0x7f3b3c0356c0 con 0x5571c5c9cb50 2024-01-28T13:03:53.762 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:53.764+0000 7f3b5825e640 1 --2- 172.21.15.31:0/3277292584 >> v2:172.21.15.31:6803/2734761187 conn(0x5571c5d2dcb0 0x5571c5dcd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:03:53.762 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:53.764+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5d1c5b0 con 0x5571c5d2dcb0 2024-01-28T13:03:53.762 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:53.764+0000 7f3b5239c640 1 --2- 172.21.15.31:0/3277292584 >> v2:172.21.15.31:6803/2734761187 conn(0x5571c5d2dcb0 0x5571c5dcd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-01-28T13:03:55.505 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:55.507+0000 7f3b5239c640 1 --2- 172.21.15.31:0/3277292584 >> v2:172.21.15.31:6803/2734761187 conn(0x5571c5d2dcb0 0x5571c5dcd340 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-28T13:03:56.952 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-01-28T13:03:56.953 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.954+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 1 ==== osd_op_reply(2 exist [writefull 0~5] v369'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.953 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.954+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5d2be30 con 0x5571c5d2dcb0 2024-01-28T13:03:56.953 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.954+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5d2c6d0 con 0x5571c5d2dcb0 2024-01-28T13:03:56.957 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.959+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 2 ==== osd_op_reply(3 exist [read 0~5 out=5b] v369'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.957 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncReadCallback (5 ms) 2024-01-28T13:03:56.957 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-01-28T13:03:56.958 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.959+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 3 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.958 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.959+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5d2cab0 con 0x5571c5d2dcb0 2024-01-28T13:03:56.958 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.959+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dcd880 con 0x5571c5d2dcb0 2024-01-28T13:03:56.961 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.963+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 4 ==== osd_op_reply(5 exist [read 0~5 out=5b] v369'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.961 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.963+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 5 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.962 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncReadFuture (4 ms) 2024-01-28T13:03:56.962 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-01-28T13:03:56.962 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.964+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5d2c210 con 0x5571c5d2dcb0 2024-01-28T13:03:56.962 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.964+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5e0e710 con 0x5571c5d2dcb0 2024-01-28T13:03:56.967 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.969+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 6 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.967 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.969+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 7 ==== osd_op_reply(7 exist [read 0~5 out=5b] v369'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.967 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncReadYield (6 ms) 2024-01-28T13:03:56.968 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-01-28T13:03:56.968 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.969+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dcebf0 con 0x5571c5d2dcb0 2024-01-28T13:03:56.977 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.979+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 8 ==== osd_op_reply(9 exist [write 0~5] v369'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:56.977 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncWriteCallback (10 ms) 2024-01-28T13:03:56.977 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-01-28T13:03:56.978 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:56.979+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dcebf0 con 0x5571c5d2dcb0 2024-01-28T13:03:57.003 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.006+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 9 ==== osd_op_reply(10 exist [write 0~5] v369'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.004 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncWriteFuture (27 ms) 2024-01-28T13:03:57.004 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-01-28T13:03:57.004 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.006+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5d19110 con 0x5571c5d2dcb0 2024-01-28T13:03:57.015 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.017+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 10 ==== osd_op_reply(11 exist [write 0~5] v369'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.015 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncWriteYield (11 ms) 2024-01-28T13:03:57.015 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-01-28T13:03:57.015 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.017+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5d19110 con 0x5571c5d2dcb0 2024-01-28T13:03:57.015 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.017+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dce670 con 0x5571c5d2dcb0 2024-01-28T13:03:57.018 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.020+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 11 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.018 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.021+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v369'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.019 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (4 ms) 2024-01-28T13:03:57.019 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-01-28T13:03:57.019 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.021+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dce1d0 con 0x5571c5d2dcb0 2024-01-28T13:03:57.019 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.021+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dce7c0 con 0x5571c5d2dcb0 2024-01-28T13:03:57.022 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.024+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 13 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.022 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.024+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 14 ==== osd_op_reply(14 exist [read 0~5 out=5b] v369'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.022 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (3 ms) 2024-01-28T13:03:57.022 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-01-28T13:03:57.023 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.024+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dce1d0 con 0x5571c5d2dcb0 2024-01-28T13:03:57.023 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.024+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5df0f50 con 0x5571c5d2dcb0 2024-01-28T13:03:57.027 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.029+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 15 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.027 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.029+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v369'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.027 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncReadOperationYield (5 ms) 2024-01-28T13:03:57.027 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-01-28T13:03:57.028 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.029+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dce1d0 con 0x5571c5d2dcb0 2024-01-28T13:03:57.038 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.040+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 17 ==== osd_op_reply(18 exist [writefull 0~5] v369'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.039 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (11 ms) 2024-01-28T13:03:57.039 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-01-28T13:03:57.039 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.041+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dce1d0 con 0x5571c5d2dcb0 2024-01-28T13:03:57.052 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.055+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 18 ==== osd_op_reply(19 exist [writefull 0~5] v369'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.053 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (14 ms) 2024-01-28T13:03:57.053 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-01-28T13:03:57.053 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.055+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 --> v2:172.21.15.31:6803/2734761187 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368) v8 -- 0x5571c5dce1d0 con 0x5571c5d2dcb0 2024-01-28T13:03:57.086 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5239c640 1 -- 172.21.15.31:0/3277292584 <== osd.1 v2:172.21.15.31:6803/2734761187 19 ==== osd_op_reply(20 exist [writefull 0~5] v369'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3b4400a430 con 0x5571c5d2dcb0 2024-01-28T13:03:57.086 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (33 ms) 2024-01-28T13:03:57.087 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 >> v2:172.21.15.31:6803/2734761187 conn(0x5571c5d2dcb0 msgr2=0x5571c5dcd340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T13:03:57.087 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 --2- 172.21.15.31:0/3277292584 >> v2:172.21.15.31:6803/2734761187 conn(0x5571c5d2dcb0 0x5571c5dcd340 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:57.087 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f3b38036310 msgr2=0x7f3b38038820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T13:03:57.087 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f3b38036310 0x7f3b38038820 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:57.087 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 msgr2=0x5571c5d18b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-28T13:03:57.087 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5d18b00 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 shutdown_connections 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 --2- 172.21.15.31:0/3277292584 >> v2:172.21.15.31:6803/2734761187 conn(0x5571c5d2dcb0 0x5571c5dcd340 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.137:6800/1465424679,v1:172.21.15.137:6801/1465424679] conn(0x7f3b38036310 0x7f3b38038820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 --2- 172.21.15.31:0/3277292584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571c5c9cb50 0x5571c5d18b00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 >> 172.21.15.31:0/3277292584 conn(0x5571c5baae00 msgr2=0x5571c5c9ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 shutdown_connections 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.088+0000 7f3b5825e640 1 -- 172.21.15.31:0/3277292584 wait complete. 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 12 tests from AsioRados (134 ms total) 2024-01-28T13:03:57.088 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:03:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:03:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 12 tests from 1 test suite ran. (4297 ms total) 2024-01-28T13:03:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 12 tests. 2024-01-28T13:03:57.089 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:03:57.089 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:03:57.089 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_list 2024-01-28T13:03:57.115 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.117+0000 7fb2a9041940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:03:57.116 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.117+0000 7fb2a9041940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:03:57.116 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:03:57.118+0000 7fb2a9041940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:03:57.116 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 11 tests from 3 test suites. 2024-01-28T13:03:57.116 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:03:57.116 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 7 tests from LibRadosList 2024-01-28T13:03:58.731 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosList.ListObjects 2024-01-28T13:04:03.606 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosList.ListObjects (4875 ms) 2024-01-28T13:04:03.606 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-01-28T13:04:03.736 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosList.ListObjectsZeroInName (130 ms) 2024-01-28T13:04:03.736 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-01-28T13:04:03.795 INFO:tasks.workunit.client.0.smithi031.stdout:myset foo1,foo2,foo3 2024-01-28T13:04:03.852 INFO:tasks.workunit.client.0.smithi031.stdout:foo1 2024-01-28T13:04:03.852 INFO:tasks.workunit.client.0.smithi031.stdout:foo2 2024-01-28T13:04:03.852 INFO:tasks.workunit.client.0.smithi031.stdout:foo3 2024-01-28T13:04:03.852 INFO:tasks.workunit.client.0.smithi031.stdout:myset foo1,foo4,foo5 2024-01-28T13:04:03.910 INFO:tasks.workunit.client.0.smithi031.stdout:foo4 2024-01-28T13:04:03.910 INFO:tasks.workunit.client.0.smithi031.stdout:foo5 2024-01-28T13:04:03.910 INFO:tasks.workunit.client.0.smithi031.stdout:foo1 2024-01-28T13:04:03.910 INFO:tasks.workunit.client.0.smithi031.stdout:myset foo6,foo7 2024-01-28T13:04:03.970 INFO:tasks.workunit.client.0.smithi031.stdout:foo7 2024-01-28T13:04:03.970 INFO:tasks.workunit.client.0.smithi031.stdout:foo6 2024-01-28T13:04:03.970 INFO:tasks.workunit.client.0.smithi031.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout:ns1:foo4 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout:ns1:foo5 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout:ns2:foo7 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout:ns2:foo6 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout:ns1:foo1 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout::foo1 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout::foo2 2024-01-28T13:04:04.027 INFO:tasks.workunit.client.0.smithi031.stdout::foo3 2024-01-28T13:04:04.136 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosList.ListObjectsNS (400 ms) 2024-01-28T13:04:04.136 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:1 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:10 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:13 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:7 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:14 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:0 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:15 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:11 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:5 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:8 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:6 0 2024-01-28T13:04:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:3 0 2024-01-28T13:04:04.307 INFO:tasks.workunit.client.0.smithi031.stdout:4 0 2024-01-28T13:04:04.307 INFO:tasks.workunit.client.0.smithi031.stdout:12 0 2024-01-28T13:04:04.307 INFO:tasks.workunit.client.0.smithi031.stdout:9 0 2024-01-28T13:04:04.307 INFO:tasks.workunit.client.0.smithi031.stdout:2 0 2024-01-28T13:04:04.361 INFO:tasks.workunit.client.0.smithi031.stdout:have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-01-28T13:04:04.523 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosList.ListObjectsStart (388 ms) 2024-01-28T13:04:04.524 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-01-28T13:04:04.630 INFO:tasks.workunit.client.0.smithi031.stdout:x cursor=MIN 2024-01-28T13:04:04.687 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-01-28T13:04:04.687 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-01-28T13:04:04.687 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-01-28T13:04:04.687 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-01-28T13:04:04.687 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-01-28T13:04:04.687 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-01-28T13:04:04.688 INFO:tasks.workunit.client.0.smithi031.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-01-28T13:04:04.743 INFO:tasks.workunit.client.0.smithi031.stdout:FIRST> seek to MIN oid=1 2024-01-28T13:04:04.799 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=1 cursor=76:02547ec2:::1:head 2024-01-28T13:04:04.800 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:02547ec2:::1:head 2024-01-28T13:04:04.855 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:02547ec2:::1:head -> 1 2024-01-28T13:04:04.855 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=10 cursor=76:52ea6a34:::10:head 2024-01-28T13:04:04.855 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:52ea6a34:::10:head 2024-01-28T13:04:04.895 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:52ea6a34:::10:head -> 10 2024-01-28T13:04:04.895 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=13 cursor=76:566253c9:::13:head 2024-01-28T13:04:04.896 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:566253c9:::13:head 2024-01-28T13:04:04.939 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:566253c9:::13:head -> 13 2024-01-28T13:04:04.939 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=7 cursor=76:5c6b0b28:::7:head 2024-01-28T13:04:04.939 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:5c6b0b28:::7:head 2024-01-28T13:04:04.975 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:5c6b0b28:::7:head -> 7 2024-01-28T13:04:04.975 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-01-28T13:04:04.975 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:62a1935d:::14:head 2024-01-28T13:04:05.014 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:62a1935d:::14:head -> 14 2024-01-28T13:04:05.014 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=0 cursor=76:6cac518f:::0:head 2024-01-28T13:04:05.014 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:6cac518f:::0:head 2024-01-28T13:04:05.056 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:6cac518f:::0:head -> 0 2024-01-28T13:04:05.056 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=15 cursor=76:863748b0:::15:head 2024-01-28T13:04:05.056 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:863748b0:::15:head 2024-01-28T13:04:05.091 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:863748b0:::15:head -> 15 2024-01-28T13:04:05.091 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=11 cursor=76:89d3ae78:::11:head 2024-01-28T13:04:05.091 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:89d3ae78:::11:head 2024-01-28T13:04:05.118 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:89d3ae78:::11:head -> 11 2024-01-28T13:04:05.118 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=5 cursor=76:b29083e3:::5:head 2024-01-28T13:04:05.119 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:b29083e3:::5:head 2024-01-28T13:04:05.156 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:b29083e3:::5:head -> 5 2024-01-28T13:04:05.157 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=8 cursor=76:bd63b0f1:::8:head 2024-01-28T13:04:05.157 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:bd63b0f1:::8:head 2024-01-28T13:04:05.176 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:bd63b0f1:::8:head -> 8 2024-01-28T13:04:05.176 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=6 cursor=76:c4fdafeb:::6:head 2024-01-28T13:04:05.176 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:c4fdafeb:::6:head 2024-01-28T13:04:05.193 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:c4fdafeb:::6:head -> 6 2024-01-28T13:04:05.193 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=3 cursor=76:cfc208b3:::3:head 2024-01-28T13:04:05.193 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:cfc208b3:::3:head 2024-01-28T13:04:05.206 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:cfc208b3:::3:head -> 3 2024-01-28T13:04:05.206 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=4 cursor=76:d83876eb:::4:head 2024-01-28T13:04:05.206 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:d83876eb:::4:head 2024-01-28T13:04:05.214 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:d83876eb:::4:head -> 4 2024-01-28T13:04:05.214 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=12 cursor=76:de5d7c5f:::12:head 2024-01-28T13:04:05.214 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:de5d7c5f:::12:head 2024-01-28T13:04:05.223 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:de5d7c5f:::12:head -> 12 2024-01-28T13:04:05.223 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=9 cursor=76:e960b815:::9:head 2024-01-28T13:04:05.223 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:e960b815:::9:head 2024-01-28T13:04:05.228 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:e960b815:::9:head -> 9 2024-01-28T13:04:05.228 INFO:tasks.workunit.client.0.smithi031.stdout:: oid=2 cursor=76:f905c69b:::2:head 2024-01-28T13:04:05.228 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:f905c69b:::2:head 2024-01-28T13:04:05.230 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:f905c69b:::2:head -> 2 2024-01-28T13:04:05.230 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:02547ec2:::1:head 2024-01-28T13:04:05.231 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:02547ec2:::1:head expected=76:02547ec2:::1:head 2024-01-28T13:04:05.286 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:02547ec2:::1:head -> 1 2024-01-28T13:04:05.286 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=1 expected=1 2024-01-28T13:04:05.286 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:c4fdafeb:::6:head 2024-01-28T13:04:05.286 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:c4fdafeb:::6:head expected=76:c4fdafeb:::6:head 2024-01-28T13:04:05.300 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:c4fdafeb:::6:head -> 6 2024-01-28T13:04:05.301 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=6 expected=6 2024-01-28T13:04:05.301 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:bd63b0f1:::8:head 2024-01-28T13:04:05.301 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:bd63b0f1:::8:head expected=76:bd63b0f1:::8:head 2024-01-28T13:04:05.319 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:bd63b0f1:::8:head -> 8 2024-01-28T13:04:05.319 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=8 expected=8 2024-01-28T13:04:05.320 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:b29083e3:::5:head 2024-01-28T13:04:05.320 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:b29083e3:::5:head expected=76:b29083e3:::5:head 2024-01-28T13:04:05.334 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:b29083e3:::5:head -> 5 2024-01-28T13:04:05.334 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=5 expected=5 2024-01-28T13:04:05.334 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:89d3ae78:::11:head 2024-01-28T13:04:05.334 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:89d3ae78:::11:head expected=76:89d3ae78:::11:head 2024-01-28T13:04:05.360 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:89d3ae78:::11:head -> 11 2024-01-28T13:04:05.360 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=11 expected=11 2024-01-28T13:04:05.360 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:566253c9:::13:head 2024-01-28T13:04:05.360 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:566253c9:::13:head expected=76:566253c9:::13:head 2024-01-28T13:04:05.399 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:566253c9:::13:head -> 13 2024-01-28T13:04:05.399 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=13 expected=13 2024-01-28T13:04:05.399 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:863748b0:::15:head 2024-01-28T13:04:05.399 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:863748b0:::15:head expected=76:863748b0:::15:head 2024-01-28T13:04:05.427 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:863748b0:::15:head -> 15 2024-01-28T13:04:05.427 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=15 expected=15 2024-01-28T13:04:05.427 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:6cac518f:::0:head 2024-01-28T13:04:05.427 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:6cac518f:::0:head expected=76:6cac518f:::0:head 2024-01-28T13:04:05.461 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:6cac518f:::0:head -> 0 2024-01-28T13:04:05.461 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=0 expected=0 2024-01-28T13:04:05.461 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:62a1935d:::14:head 2024-01-28T13:04:05.461 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:62a1935d:::14:head expected=76:62a1935d:::14:head 2024-01-28T13:04:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:62a1935d:::14:head -> 14 2024-01-28T13:04:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=14 expected=14 2024-01-28T13:04:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:5c6b0b28:::7:head 2024-01-28T13:04:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:5c6b0b28:::7:head expected=76:5c6b0b28:::7:head 2024-01-28T13:04:05.533 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:5c6b0b28:::7:head -> 7 2024-01-28T13:04:05.533 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=7 expected=7 2024-01-28T13:04:05.533 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:52ea6a34:::10:head 2024-01-28T13:04:05.533 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:52ea6a34:::10:head expected=76:52ea6a34:::10:head 2024-01-28T13:04:05.572 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:52ea6a34:::10:head -> 10 2024-01-28T13:04:05.573 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=10 expected=10 2024-01-28T13:04:05.573 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:f905c69b:::2:head 2024-01-28T13:04:05.573 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:f905c69b:::2:head expected=76:f905c69b:::2:head 2024-01-28T13:04:05.574 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:f905c69b:::2:head -> 2 2024-01-28T13:04:05.574 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=2 expected=2 2024-01-28T13:04:05.574 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:e960b815:::9:head 2024-01-28T13:04:05.574 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:e960b815:::9:head expected=76:e960b815:::9:head 2024-01-28T13:04:05.579 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:e960b815:::9:head -> 9 2024-01-28T13:04:05.579 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=9 expected=9 2024-01-28T13:04:05.579 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:de5d7c5f:::12:head 2024-01-28T13:04:05.579 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:de5d7c5f:::12:head expected=76:de5d7c5f:::12:head 2024-01-28T13:04:05.588 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:de5d7c5f:::12:head -> 12 2024-01-28T13:04:05.588 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=12 expected=12 2024-01-28T13:04:05.588 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:d83876eb:::4:head 2024-01-28T13:04:05.588 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:d83876eb:::4:head expected=76:d83876eb:::4:head 2024-01-28T13:04:05.596 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:d83876eb:::4:head -> 4 2024-01-28T13:04:05.596 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=4 expected=4 2024-01-28T13:04:05.596 INFO:tasks.workunit.client.0.smithi031.stdout:: seek to 76:cfc208b3:::3:head 2024-01-28T13:04:05.597 INFO:tasks.workunit.client.0.smithi031.stdout:: cursor()=76:cfc208b3:::3:head expected=76:cfc208b3:::3:head 2024-01-28T13:04:05.609 INFO:tasks.workunit.client.0.smithi031.stdout:> 76:cfc208b3:::3:head -> 3 2024-01-28T13:04:05.609 INFO:tasks.workunit.client.0.smithi031.stdout:: entry=3 expected=3 2024-01-28T13:04:06.332 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosList.ListObjectsCursor (1808 ms) 2024-01-28T13:04:06.332 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-01-28T13:04:06.658 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosList.EnumerateObjects (326 ms) 2024-01-28T13:04:06.658 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-01-28T13:04:06.766 INFO:tasks.workunit.client.0.smithi031.stdout:split 0/5 -> MIN 76:33333333::::head 2024-01-28T13:04:06.779 INFO:tasks.workunit.client.0.smithi031.stdout:split 1/5 -> 76:33333333::::head 76:66666666::::head 2024-01-28T13:04:06.791 INFO:tasks.workunit.client.0.smithi031.stdout:split 2/5 -> 76:66666666::::head 76:99999999::::head 2024-01-28T13:04:06.805 INFO:tasks.workunit.client.0.smithi031.stdout:split 3/5 -> 76:99999999::::head 76:cccccccc::::head 2024-01-28T13:04:06.817 INFO:tasks.workunit.client.0.smithi031.stdout:split 4/5 -> 76:cccccccc::::head MAX 2024-01-28T13:04:07.692 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosList.EnumerateObjectsSplit (1034 ms) 2024-01-28T13:04:07.723 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 7 tests from LibRadosList (8962 ms total) 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosListEC 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:70: Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:84: Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:96: Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:84: Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.724 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:96: Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:84: Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:96: Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:78: Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosListNP 2024-01-28T13:04:07.725 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosListNP.ListObjectsError (7026 ms) 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosListNP (7026 ms total) 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 11 tests from 3 test suites ran. (17632 ms total) 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 8 tests. 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 3 tests, listed below: 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-01-28T13:04:14.748 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-01-28T13:04:14.749 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-01-28T13:04:14.750 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:04:14.750 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:04:14.750 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_lock 2024-01-28T13:04:14.767 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:04:14.767 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 16 tests from 2 test suites. 2024-01-28T13:04:14.767 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:04:14.767 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLock 2024-01-28T13:04:16.754 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.LockExclusive 2024-01-28T13:04:21.125 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.LockExclusive (4371 ms) 2024-01-28T13:04:21.125 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.LockShared 2024-01-28T13:04:26.322 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.LockShared (5196 ms) 2024-01-28T13:04:26.322 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.LockExclusiveDur 2024-01-28T13:04:32.554 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.LockExclusiveDur (6232 ms) 2024-01-28T13:04:32.554 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.LockSharedDur 2024-01-28T13:04:35.321 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.LockSharedDur (2768 ms) 2024-01-28T13:04:35.321 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.LockMayRenew 2024-01-28T13:04:37.234 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.LockMayRenew (1913 ms) 2024-01-28T13:04:37.234 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.Unlock 2024-01-28T13:04:37.261 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.Unlock (27 ms) 2024-01-28T13:04:37.261 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.ListLockers 2024-01-28T13:04:37.297 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.ListLockers (36 ms) 2024-01-28T13:04:37.297 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLock.BreakLock 2024-01-28T13:04:37.320 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLock.BreakLock (24 ms) 2024-01-28T13:04:37.729 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLock (20567 ms total) 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLockEC 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:168: Skipped 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.LockExclusive 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive (0 ms) 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.LockShared 2024-01-28T13:04:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockShared (0 ms) 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.LockExclusiveDur 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur (0 ms) 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.LockSharedDur 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur (0 ms) 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.LockMayRenew 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew (0 ms) 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.Unlock 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.Unlock (0 ms) 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.ListLockers 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers (0 ms) 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockEC.BreakLock 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock (0 ms) 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:176: Skipped 2024-01-28T13:04:37.733 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLockEC (0 ms total) 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 16 tests from 2 test suites ran. (22961 ms total) 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 8 tests. 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 8 tests, listed below: 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockShared 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.Unlock 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers 2024-01-28T13:04:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock 2024-01-28T13:04:37.735 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:04:37.735 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:04:37.735 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_lock_pp 2024-01-28T13:04:37.747 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:04:37.747 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 16 tests from 2 test suites. 2024-01-28T13:04:37.747 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:04:37.747 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLockPP 2024-01-28T13:04:37.747 INFO:tasks.workunit.client.0.smithi031.stdout:seed 33514 2024-01-28T13:04:39.739 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.LockExclusivePP 2024-01-28T13:04:44.684 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.LockExclusivePP (4945 ms) 2024-01-28T13:04:44.684 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.LockSharedPP 2024-01-28T13:04:44.701 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.LockSharedPP (17 ms) 2024-01-28T13:04:44.701 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-01-28T13:04:47.129 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.LockExclusiveDurPP (2428 ms) 2024-01-28T13:04:47.129 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.LockSharedDurPP 2024-01-28T13:04:48.153 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.LockSharedDurPP (1024 ms) 2024-01-28T13:04:48.153 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.LockMayRenewPP 2024-01-28T13:04:48.180 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.LockMayRenewPP (27 ms) 2024-01-28T13:04:48.180 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.UnlockPP 2024-01-28T13:04:48.208 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.UnlockPP (28 ms) 2024-01-28T13:04:48.209 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.ListLockersPP 2024-01-28T13:04:48.243 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.ListLockersPP (34 ms) 2024-01-28T13:04:48.243 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockPP.BreakLockPP 2024-01-28T13:04:48.271 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosLockPP.BreakLockPP (28 ms) 2024-01-28T13:04:48.756 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLockPP (8532 ms total) 2024-01-28T13:04:48.756 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLockECPP 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.LockExclusivePP 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP (0 ms) 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.LockSharedPP 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.757 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP (0 ms) 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP (0 ms) 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP (0 ms) 2024-01-28T13:04:48.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP (0 ms) 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.UnlockPP 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP (0 ms) 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.ListLockersPP 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP (0 ms) 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosLockECPP.BreakLockPP 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP (0 ms) 2024-01-28T13:04:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:04:48.761 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:04:48.761 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosLockECPP (0 ms total) 2024-01-28T13:04:48.761 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 16 tests from 2 test suites ran. (11009 ms total) 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 8 tests. 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 8 tests, listed below: 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP 2024-01-28T13:04:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP 2024-01-28T13:04:48.763 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:04:48.763 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:04:48.763 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_misc 2024-01-28T13:04:48.789 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:04:48.791+0000 7f104548b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:04:48.789 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:04:48.791+0000 7f104548b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:04:48.789 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:04:48.791+0000 7f104548b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:04:48.789 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 12 tests from 4 test suites. 2024-01-28T13:04:48.789 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:04:48.789 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscVersion 2024-01-28T13:04:48.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscVersion.Version 2024-01-28T13:04:48.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscVersion.Version (0 ms) 2024-01-28T13:04:48.790 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-01-28T13:04:48.790 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:48.790 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosMiscConnectFailure 2024-01-28T13:04:48.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-01-28T13:04:48.816 INFO:tasks.workunit.client.0.smithi031.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2024-01-28T13:04:48.816 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:04:48.819+0000 7f104548b640 -1 failed for service _ceph-mon._tcp 2024-01-28T13:04:48.817 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:04:48.819+0000 7f104548b640 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-01-28T13:04:48.817 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectFailure (28 ms) 2024-01-28T13:04:48.817 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-01-28T13:04:50.823 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2006 ms) 2024-01-28T13:04:50.823 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosMiscConnectFailure (2034 ms total) 2024-01-28T13:04:50.823 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:04:50.823 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscPool 2024-01-28T13:04:50.823 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPool.PoolCreationRace 2024-01-28T13:04:53.770 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x7f100c005d20 2024-01-28T13:04:54.770 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x7f102c005d20 2024-01-28T13:04:55.770 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d6b41a0 2024-01-28T13:04:56.770 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d6a6e50 2024-01-28T13:04:57.770 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d6c8050 2024-01-28T13:04:58.770 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9e9c90 2024-01-28T13:04:59.771 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9ea740 2024-01-28T13:05:00.771 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9eb1a0 2024-01-28T13:05:01.771 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9ebba0 2024-01-28T13:05:02.771 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9ec250 2024-01-28T13:05:03.771 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9ec900 2024-01-28T13:05:04.771 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9ed380 2024-01-28T13:05:05.771 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9eb920 2024-01-28T13:05:06.772 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9ee2c0 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:started 0x55e02d9ed130 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout: started 15 aios 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x7f100c005d20 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x7f102c005d20 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d6b41a0 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d6a6e50 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d6c8050 2024-01-28T13:05:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9e9c90 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9ea740 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9eb1a0 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9ebba0 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9ec250 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9ec900 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9ed380 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9eb920 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9ee2c0 2024-01-28T13:05:07.773 INFO:tasks.workunit.client.0.smithi031.stdout:waiting 0x55e02d9ed130 2024-01-28T13:05:09.153 INFO:tasks.workunit.client.0.smithi031.stdout:done. 2024-01-28T13:05:10.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPool.PoolCreationRace (19947 ms) 2024-01-28T13:05:10.770 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscPool (19947 ms total) 2024-01-28T13:05:10.770 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:05:10.770 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosMisc 2024-01-28T13:05:12.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.ClusterFSID 2024-01-28T13:05:12.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-01-28T13:05:12.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.Exec 2024-01-28T13:05:33.201 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.Exec (20421 ms) 2024-01-28T13:05:33.201 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.WriteSame 2024-01-28T13:05:33.227 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.WriteSame (26 ms) 2024-01-28T13:05:33.227 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.CmpExt 2024-01-28T13:05:33.259 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.CmpExt (32 ms) 2024-01-28T13:05:33.259 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.Applications 2024-01-28T13:05:37.752 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.Applications (4493 ms) 2024-01-28T13:05:37.752 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.MinCompatOSD 2024-01-28T13:05:37.753 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-01-28T13:05:37.753 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.MinCompatClient 2024-01-28T13:05:37.753 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-01-28T13:05:37.753 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMisc.ShutdownRace 2024-01-28T13:06:00.879 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMisc.ShutdownRace (23127 ms) 2024-01-28T13:06:01.768 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 8 tests from LibRadosMisc (48099 ms total) 2024-01-28T13:06:01.768 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:06:01.768 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:06:01.768 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 12 tests from 4 test suites ran. (72979 ms total) 2024-01-28T13:06:01.768 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 12 tests. 2024-01-28T13:06:01.784 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:06:01.784 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:06:01.784 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_misc_pp 2024-01-28T13:06:01.814 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:06:01.816+0000 7fcae3a347c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:06:01.814 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:06:01.816+0000 7fcae3a347c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:06:01.815 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:06:01.816+0000 7fcae3a347c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:06:01.815 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 31 tests from 7 test suites. 2024-01-28T13:06:01.815 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:06:01.815 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscVersion 2024-01-28T13:06:01.815 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscVersion.VersionPP 2024-01-28T13:06:01.815 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-01-28T13:06:01.816 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-01-28T13:06:01.816 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:06:01.816 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 22 tests from LibRadosMiscPP 2024-01-28T13:06:01.816 INFO:tasks.workunit.client.0.smithi031.stdout:seed 95180 2024-01-28T13:06:03.768 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-01-28T13:06:03.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-01-28T13:06:03.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.LongNamePP 2024-01-28T13:06:13.612 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.LongNamePP (9843 ms) 2024-01-28T13:06:13.612 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.LongLocatorPP 2024-01-28T13:06:15.219 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.LongLocatorPP (1607 ms) 2024-01-28T13:06:15.220 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.LongNSpacePP 2024-01-28T13:06:15.255 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.LongNSpacePP (36 ms) 2024-01-28T13:06:15.255 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-01-28T13:06:15.318 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.LongAttrNamePP (63 ms) 2024-01-28T13:06:15.318 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.ExecPP 2024-01-28T13:06:15.340 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.ExecPP (22 ms) 2024-01-28T13:06:15.340 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.BadFlagsPP 2024-01-28T13:06:15.354 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.BadFlagsPP (14 ms) 2024-01-28T13:06:15.354 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.Operate1PP 2024-01-28T13:06:17.855 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.Operate1PP (2501 ms) 2024-01-28T13:06:17.855 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.Operate2PP 2024-01-28T13:06:18.144 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.Operate2PP (289 ms) 2024-01-28T13:06:18.144 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.BigObjectPP 2024-01-28T13:06:18.175 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.BigObjectPP (31 ms) 2024-01-28T13:06:18.175 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.AioOperatePP 2024-01-28T13:06:18.184 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.AioOperatePP (10 ms) 2024-01-28T13:06:18.185 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.AssertExistsPP 2024-01-28T13:06:18.199 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.AssertExistsPP (14 ms) 2024-01-28T13:06:18.199 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.AssertVersionPP 2024-01-28T13:06:18.235 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.AssertVersionPP (34 ms) 2024-01-28T13:06:18.235 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.BigAttrPP 2024-01-28T13:06:18.239 INFO:tasks.workunit.client.0.smithi031.stdout:osd_max_attr_size = 0 2024-01-28T13:06:18.240 INFO:tasks.workunit.client.0.smithi031.stdout:osd_max_attr_size == 0; skipping test 2024-01-28T13:06:27.722 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.BigAttrPP (9488 ms) 2024-01-28T13:06:27.722 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.CopyPP 2024-01-28T13:06:27.722 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/misc_cxx.cc:386: Skipped 2024-01-28T13:06:27.722 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:06:27.722 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP (0 ms) 2024-01-28T13:06:27.723 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.CopyScrubPP 2024-01-28T13:06:27.723 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/misc_cxx.cc:549: Skipped 2024-01-28T13:06:27.723 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:06:27.723 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP (0 ms) 2024-01-28T13:06:27.723 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.WriteSamePP 2024-01-28T13:06:27.744 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.WriteSamePP (23 ms) 2024-01-28T13:06:27.744 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.CmpExtPP 2024-01-28T13:06:27.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.CmpExtPP (15 ms) 2024-01-28T13:06:27.758 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.Applications 2024-01-28T13:06:32.769 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.Applications (5010 ms) 2024-01-28T13:06:32.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.MinCompatOSD 2024-01-28T13:06:32.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-01-28T13:06:32.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.MinCompatClient 2024-01-28T13:06:32.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-01-28T13:06:32.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscPP.Conf 2024-01-28T13:06:32.770 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosMiscPP.Conf (0 ms) 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 22 tests from LibRadosMiscPP (29001 ms total) 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosTwoPoolsECPP 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/misc_cxx.cc:476: Skipped 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/misc_cxx.cc:498: Skipped 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/misc_cxx.cc:504: Skipped 2024-01-28T13:06:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:06:33.774 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom (0 ms) 2024-01-28T13:06:33.774 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/misc_cxx.cc:491: Skipped 2024-01-28T13:06:33.774 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:06:33.774 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosTwoPoolsECPP (0 ms total) 2024-01-28T13:06:33.774 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:06:33.774 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-01-28T13:06:35.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosChecksum/0.Subset 2024-01-28T13:06:51.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosChecksum/0.Subset (15995 ms) 2024-01-28T13:06:51.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosChecksum/0.Chunked 2024-01-28T13:06:55.132 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosChecksum/0.Chunked (3355 ms) 2024-01-28T13:06:55.755 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosChecksum/0 (19350 ms total) 2024-01-28T13:06:55.755 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:06:55.755 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-01-28T13:06:57.767 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosChecksum/1.Subset 2024-01-28T13:07:15.182 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosChecksum/1.Subset (17415 ms) 2024-01-28T13:07:15.182 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosChecksum/1.Chunked 2024-01-28T13:07:15.864 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosChecksum/1.Chunked (682 ms) 2024-01-28T13:07:16.774 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosChecksum/1 (18097 ms total) 2024-01-28T13:07:16.774 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:07:16.774 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-01-28T13:07:18.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosChecksum/2.Subset 2024-01-28T13:07:24.743 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosChecksum/2.Subset (5962 ms) 2024-01-28T13:07:24.743 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosChecksum/2.Chunked 2024-01-28T13:07:31.423 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosChecksum/2.Chunked (6680 ms) 2024-01-28T13:07:31.906 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosChecksum/2 (12642 ms total) 2024-01-28T13:07:31.906 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscECPP 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange (0 ms) 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosMiscECPP (0 ms total) 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:07:31.907 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 31 tests from 7 test suites ran. (90094 ms total) 2024-01-28T13:07:31.908 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 27 tests. 2024-01-28T13:07:31.908 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 4 tests, listed below: 2024-01-28T13:07:31.908 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP 2024-01-28T13:07:31.908 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP 2024-01-28T13:07:31.908 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom 2024-01-28T13:07:31.908 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange 2024-01-28T13:07:31.909 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:07:31.909 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:07:31.909 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_tier_pp 2024-01-28T13:07:31.941 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:07:31.945+0000 7f16ef85b940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:07:31.941 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:07:31.945+0000 7f16ef85b940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:07:31.941 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:07:31.945+0000 7f16ef85b940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:07:31.942 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 77 tests from 4 test suites. 2024-01-28T13:07:31.942 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:07:31.942 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosTierPP 2024-01-28T13:07:31.942 INFO:tasks.workunit.client.0.smithi031.stdout:seed 95253 2024-01-28T13:07:33.918 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTierPP.Dirty 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:287: Skipped 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierPP.Dirty (0 ms) 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTierPP.FlushWriteRaces 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:2182: Skipped 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces (0 ms) 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTierPP.HitSetNone 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:2561: Skipped 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone (1 ms) 2024-01-28T13:07:34.919 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosTierPP (1 ms total) 2024-01-28T13:07:34.919 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:07:34.919 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 48 tests from LibRadosTwoPoolsPP 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.Overlay 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay (0 ms) 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.Promote 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote (0 ms) 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap (0 ms) 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub (0 ms) 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (0 ms) 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout (0 ms) 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (0 ms) 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.Evict 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict (0 ms) 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap (1 ms) 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 (0 ms) 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap (0 ms) 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-01-28T13:07:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (0 ms) 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush (0 ms) 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.Flush 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush (0 ms) 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap (0 ms) 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces (0 ms) 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-01-28T13:07:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace (0 ms) 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead (0 ms) 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite (0 ms) 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim (0 ms) 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead (0 ms) 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-01-28T13:07:36.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead (0 ms) 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.CachePin 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin (0 ms) 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.785 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead (0 ms) 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead (0 ms) 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.786 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead (0 ms) 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset (0 ms) 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead (0 ms) 2024-01-28T13:07:36.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount (0 ms) 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (0 ms) 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.788 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (0 ms) 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (0 ms) 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (0 ms) 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount (0 ms) 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict (0 ms) 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.790 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote (0 ms) 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (0 ms) 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead (0 ms) 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap (0 ms) 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount (0 ms) 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush (0 ms) 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (0 ms) 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.793 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback (0 ms) 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (0 ms) 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback (0 ms) 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.794 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError (0 ms) 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn (0 ms) 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:223: Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:233: Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (0 ms) 2024-01-28T13:07:37.773 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 48 tests from LibRadosTwoPoolsPP (1 ms total) 2024-01-28T13:07:37.773 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:07:37.773 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosTierECPP 2024-01-28T13:07:37.773 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:07:37.773 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.773 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTierECPP.Dirty 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty (0 ms) 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces (0 ms) 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTierECPP.CallForcesPromote 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote (0 ms) 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTierECPP.HitSetNone 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone (0 ms) 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosTierECPP (0 ms total) 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6267: Skipped 2024-01-28T13:07:37.775 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay (0 ms) 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.Promote 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote (0 ms) 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-01-28T13:07:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap (0 ms) 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (0 ms) 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout (0 ms) 2024-01-28T13:07:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict (0 ms) 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap (0 ms) 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.778 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush (0 ms) 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush (0 ms) 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush (0 ms) 2024-01-28T13:07:37.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap (0 ms) 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (0 ms) 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace (0 ms) 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead (0 ms) 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim (0 ms) 2024-01-28T13:07:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (0 ms) 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead (0 ms) 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin (0 ms) 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead (0 ms) 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead (0 ms) 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.783 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead (0 ms) 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6278: Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6288: Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier (0 ms) 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/tier_cxx.cc:6272: Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP (1 ms total) 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:07:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 77 tests from 4 test suites ran. (5828 ms total) 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 0 tests. 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 77 tests, listed below: 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierPP.Dirty 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 2024-01-28T13:07:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset 2024-01-28T13:07:37.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-01-28T13:07:37.787 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap 2024-01-28T13:07:37.788 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-01-28T13:07:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_pool 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 9 tests from 1 test suite. 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 9 tests from LibRadosPools 2024-01-28T13:07:37.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolList 2024-01-28T13:07:40.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolList (2994 ms) 2024-01-28T13:07:40.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolLookup 2024-01-28T13:07:43.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolLookup (3010 ms) 2024-01-28T13:07:43.791 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolLookup2 2024-01-28T13:07:46.765 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolLookup2 (2974 ms) 2024-01-28T13:07:46.765 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-01-28T13:07:49.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolLookupOtherInstance (3019 ms) 2024-01-28T13:07:49.784 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-01-28T13:07:52.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3005 ms) 2024-01-28T13:07:52.789 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolDelete 2024-01-28T13:07:57.774 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolDelete (4985 ms) 2024-01-28T13:07:57.774 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolCreateDelete 2024-01-28T13:08:02.785 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolCreateDelete (5012 ms) 2024-01-28T13:08:02.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPools.PoolCreateWithCrushRule (4989 ms) 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPools.PoolGetBaseTier 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/pool.cc:139: Skipped 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier (0 ms) 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 9 tests from LibRadosPools (29988 ms total) 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 9 tests from 1 test suite ran. (29988 ms total) 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 8 tests. 2024-01-28T13:08:07.775 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 1 test, listed below: 2024-01-28T13:08:07.776 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier 2024-01-28T13:08:07.777 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:08:07.777 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:08:07.777 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_snapshots 2024-01-28T13:08:07.792 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:08:07.792 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 13 tests from 4 test suites. 2024-01-28T13:08:07.792 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:08:07.792 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosSnapshots 2024-01-28T13:08:09.786 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshots.SnapList 2024-01-28T13:08:56.040 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshots.SnapList (46254 ms) 2024-01-28T13:08:56.040 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshots.SnapRemove 2024-01-28T13:09:01.054 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshots.SnapRemove (5015 ms) 2024-01-28T13:09:01.055 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshots.Rollback 2024-01-28T13:09:04.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshots.Rollback (3736 ms) 2024-01-28T13:09:04.790 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshots.SnapGetName 2024-01-28T13:09:06.779 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshots.SnapGetName (1989 ms) 2024-01-28T13:09:07.782 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosSnapshots (56994 ms total) 2024-01-28T13:09:07.782 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:09:07.782 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-01-28T13:09:09.798 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-01-28T13:09:33.888 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Snap (24090 ms) 2024-01-28T13:09:33.889 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-01-28T13:09:44.684 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Rollback (10796 ms) 2024-01-28T13:09:44.684 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-01-28T13:09:54.307 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (9623 ms) 2024-01-28T13:09:54.820 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged (44509 ms total) 2024-01-28T13:09:54.820 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:09:54.820 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosSnapshotsEC 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:168: Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsEC.SnapList 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList (0 ms) 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove (0 ms) 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsEC.Rollback 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback (0 ms) 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName (0 ms) 2024-01-28T13:09:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:176: Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosSnapshotsEC (0 ms total) 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:168: Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap (0 ms) 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-01-28T13:09:54.823 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback (0 ms) 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:176: Skipped 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC (0 ms total) 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 13 tests from 4 test suites ran. (107028 ms total) 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 7 tests. 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 6 tests, listed below: 2024-01-28T13:09:54.824 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:09:54.825 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_snapshots_pp 2024-01-28T13:09:54.842 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:09:54.842 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 21 tests from 5 test suites. 2024-01-28T13:09:54.842 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:09:54.842 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 5 tests from LibRadosSnapshotsPP 2024-01-28T13:09:54.842 INFO:tasks.workunit.client.0.smithi031.stdout:seed 95509 2024-01-28T13:09:56.793 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-01-28T13:10:05.798 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsPP.SnapListPP (9004 ms) 2024-01-28T13:10:05.798 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-01-28T13:10:12.829 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsPP.SnapRemovePP (7032 ms) 2024-01-28T13:10:12.829 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-01-28T13:10:22.806 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsPP.RollbackPP (9977 ms) 2024-01-28T13:10:22.806 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-01-28T13:10:25.819 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsPP.SnapGetNamePP (3014 ms) 2024-01-28T13:10:25.820 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-01-28T13:10:33.845 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (8025 ms) 2024-01-28T13:10:34.842 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 5 tests from LibRadosSnapshotsPP (37052 ms total) 2024-01-28T13:10:34.842 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:10:34.842 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-01-28T13:10:36.851 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-01-28T13:10:39.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-01-28T13:10:45.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~6s 2024-01-28T13:10:50.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~11s 2024-01-28T13:10:56.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~17s 2024-01-28T13:11:01.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~22s 2024-01-28T13:11:07.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~28s 2024-01-28T13:11:12.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~33s 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (36810 ms) 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/snapshots_cxx.cc:156: Skipped 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP (0 ms) 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/snapshots_cxx.cc:242: Skipped 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (0 ms) 2024-01-28T13:11:13.662 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-01-28T13:11:18.342 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~39s 2024-01-28T13:11:18.820 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (5159 ms) 2024-01-28T13:11:18.820 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-01-28T13:11:23.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~44s 2024-01-28T13:11:24.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (5961 ms) 2024-01-28T13:11:24.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-01-28T13:11:24.781 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-01-28T13:11:24.781 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:11:24.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-01-28T13:11:24.781 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-01-28T13:11:27.070 INFO:tasks.workunit.client.0.smithi031.stdout:deleting snap 9 in pool LibRadosSnapshotsSelfManagedPP_smithi031-95509-7 2024-01-28T13:11:28.071 INFO:tasks.workunit.client.0.smithi031.stdout:waiting for snaps to purge 2024-01-28T13:11:29.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2024-01-28T13:11:34.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~55s 2024-01-28T13:11:40.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~61s 2024-01-28T13:11:43.083 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18303 ms) 2024-01-28T13:11:44.070 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP (66233 ms total) 2024-01-28T13:11:44.070 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:11:44.070 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-01-28T13:11:44.070 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-01-28T13:11:44.076 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (6 ms) 2024-01-28T13:11:44.076 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-01-28T13:11:45.866 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~66s 2024-01-28T13:11:49.127 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5050 ms) 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5056 ms total) 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosSnapshotsECPP 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP (0 ms) 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP (0 ms) 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP (0 ms) 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-01-28T13:11:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP (1 ms) 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosSnapshotsECPP (1 ms total) 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:11:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP (0 ms) 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (0 ms) 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (0 ms) 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (0 ms total) 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 21 tests from 5 test suites ran. (114286 ms total) 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 11 tests. 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 10 tests, listed below: 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-01-28T13:11:49.132 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-01-28T13:11:49.133 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:11:49.134 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:11:49.134 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_stat 2024-01-28T13:11:49.145 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:11:49.145 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 9 tests from 2 test suites. 2024-01-28T13:11:49.145 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:11:49.145 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 5 tests from LibRadosStat 2024-01-28T13:11:50.809 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStat.Stat 2024-01-28T13:11:51.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~72s 2024-01-28T13:11:56.875 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~77s 2024-01-28T13:12:02.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~83s 2024-01-28T13:12:06.585 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStat.Stat (15776 ms) 2024-01-28T13:12:06.585 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStat.Stat2 2024-01-28T13:12:06.641 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStat.Stat2 (56 ms) 2024-01-28T13:12:06.641 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStat.StatNS 2024-01-28T13:12:07.884 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~88s 2024-01-28T13:12:08.657 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStat.StatNS (2015 ms) 2024-01-28T13:12:08.657 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStat.ClusterStat 2024-01-28T13:12:08.657 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStat.ClusterStat (1 ms) 2024-01-28T13:12:08.657 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStat.PoolStat 2024-01-28T13:12:08.665 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStat.PoolStat (8 ms) 2024-01-28T13:12:08.827 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 5 tests from LibRadosStat (17856 ms total) 2024-01-28T13:12:08.827 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosStatEC 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:168: Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatEC.Stat 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.Stat (0 ms) 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatEC.StatNS 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:12:08.828 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.StatNS (0 ms) 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatEC.ClusterStat 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat (0 ms) 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatEC.PoolStat 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat (0 ms) 2024-01-28T13:12:08.829 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:176: Skipped 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosStatEC (0 ms total) 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 9 tests from 2 test suites ran. (19682 ms total) 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 5 tests. 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 4 tests, listed below: 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.Stat 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.StatNS 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat 2024-01-28T13:12:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat 2024-01-28T13:12:08.831 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:12:08.831 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:12:08.831 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_stat_pp 2024-01-28T13:12:08.843 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:12:08.844 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 9 tests from 2 test suites. 2024-01-28T13:12:08.844 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:12:08.844 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 5 tests from LibRadosStatPP 2024-01-28T13:12:08.844 INFO:tasks.workunit.client.0.smithi031.stdout:seed 95646 2024-01-28T13:12:10.816 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatPP.StatPP 2024-01-28T13:12:13.389 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~94s 2024-01-28T13:12:15.821 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStatPP.StatPP (5005 ms) 2024-01-28T13:12:15.821 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-01-28T13:12:15.888 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStatPP.Stat2Mtime2PP (67 ms) 2024-01-28T13:12:15.888 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatPP.ClusterStatPP 2024-01-28T13:12:15.888 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-01-28T13:12:15.888 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatPP.PoolStatPP 2024-01-28T13:12:15.912 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStatPP.PoolStatPP (24 ms) 2024-01-28T13:12:15.912 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatPP.StatPPNS 2024-01-28T13:12:15.941 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosStatPP.StatPPNS (29 ms) 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 5 tests from LibRadosStatPP (5125 ms total) 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosStatECPP 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatECPP.StatPP 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:12:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP (0 ms) 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatECPP.ClusterStatPP 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatECPP.PoolStatPP 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP (0 ms) 2024-01-28T13:12:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosStatECPP.StatPPNS 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS (0 ms) 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosStatECPP (0 ms total) 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 9 tests from 2 test suites ran. (7235 ms total) 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 5 tests. 2024-01-28T13:12:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 4 tests, listed below: 2024-01-28T13:12:16.083 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP 2024-01-28T13:12:16.083 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP 2024-01-28T13:12:16.083 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP 2024-01-28T13:12:16.083 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS 2024-01-28T13:12:16.083 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:12:16.083 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:12:16.083 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_watch_notify 2024-01-28T13:12:16.096 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:12:16.096 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 11 tests from 2 test suites. 2024-01-28T13:12:16.096 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:12:16.096 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 10 tests from LibRadosWatchNotify 2024-01-28T13:12:18.087 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify 2024-01-28T13:12:18.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~99s 2024-01-28T13:12:24.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~105s 2024-01-28T13:12:29.903 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~110s 2024-01-28T13:12:34.528 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify_test_cb 2024-01-28T13:12:34.637 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.WatchNotify (16550 ms) 2024-01-28T13:12:34.637 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.Watch2Delete 2024-01-28T13:12:35.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~116s 2024-01-28T13:12:38.056 INFO:tasks.workunit.client.0.smithi031.stdout:waiting up to 300 for disconnect notification ... 2024-01-28T13:12:40.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~121s 2024-01-28T13:12:41.122 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_errcb cookie 94412244958048 err -107 2024-01-28T13:12:42.846 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.Watch2Delete (8210 ms) 2024-01-28T13:12:42.847 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-01-28T13:12:42.980 INFO:tasks.workunit.client.0.smithi031.stdout:waiting up to 300 for disconnect notification ... 2024-01-28T13:12:46.123 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_errcb cookie 94412244958048 err -107 2024-01-28T13:12:46.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~127s 2024-01-28T13:12:46.987 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete (4140 ms) 2024-01-28T13:12:46.987 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2 2024-01-28T13:12:47.003 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597761 cookie 94412244958048 2024-01-28T13:12:47.015 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2 (29 ms) 2024-01-28T13:12:47.015 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-01-28T13:12:47.029 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597762 cookie 94412244958048 2024-01-28T13:12:47.044 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.AioWatchNotify2 (29 ms) 2024-01-28T13:12:47.044 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.AioNotify 2024-01-28T13:12:47.057 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597763 cookie 94412244958048 2024-01-28T13:12:47.068 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.AioNotify (24 ms) 2024-01-28T13:12:47.068 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-01-28T13:12:47.093 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597760 cookie 94412244974416 2024-01-28T13:12:47.093 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597760 cookie 94412244975536 2024-01-28T13:12:47.109 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Multi (41 ms) 2024-01-28T13:12:47.109 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-01-28T13:12:47.127 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597760 cookie 94412244974416 2024-01-28T13:12:50.128 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597761 cookie 94412244974416 2024-01-28T13:12:50.144 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3034 ms) 2024-01-28T13:12:50.144 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-01-28T13:12:50.163 INFO:tasks.workunit.client.0.smithi031.stdout:waiting up to 1024 for osd to time us out ... 2024-01-28T13:12:52.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~132s 2024-01-28T13:12:54.171 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_errcb cookie 94412244974416 err -107 2024-01-28T13:12:55.185 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_cb from 15523 notify_id 2942052597765 cookie 94412244974416 2024-01-28T13:12:55.196 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.Watch3Timeout (5053 ms) 2024-01-28T13:12:55.197 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-01-28T13:12:55.214 INFO:tasks.workunit.client.0.smithi031.stdout:waiting up to 30 for disconnect notification ... 2024-01-28T13:12:56.125 INFO:tasks.workunit.client.0.smithi031.stdout:watch_notify2_test_errcb cookie 94412244974416 err -107 2024-01-28T13:12:56.220 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete2 (1024 ms) 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 10 tests from LibRadosWatchNotify (38134 ms total) 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosWatchNotifyEC 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:168: Skipped 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:182: Skipped 2024-01-28T13:12:57.089 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:196: Skipped 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify (0 ms) 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/TestCase.cc:176: Skipped 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 1 test from LibRadosWatchNotifyEC (0 ms total) 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 11 tests from 2 test suites ran. (40993 ms total) 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 10 tests. 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 1 test, listed below: 2024-01-28T13:12:57.090 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify 2024-01-28T13:12:57.091 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:12:57.091 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:12:57.091 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 16 tests from 2 test suites. 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:373: Skipped 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:12:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify (0 ms) 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:387: Skipped 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:400: Skipped 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (0 ms) 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/testcase_cxx.cc:381: Skipped 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:Not supported by crimson yet. Skipped 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP (0 ms total) 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:12:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-01-28T13:12:57.539 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~138s 2024-01-28T13:12:59.096 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-01-28T13:13:03.044 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~143s 2024-01-28T13:13:08.550 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~149s 2024-01-28T13:13:14.055 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~154s 2024-01-28T13:13:14.660 INFO:tasks.workunit.client.0.smithi031.stdout:notify 2024-01-28T13:13:14.671 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (15575 ms) 2024-01-28T13:13:14.671 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-01-28T13:13:14.701 INFO:tasks.workunit.client.0.smithi031.stdout:notify 2024-01-28T13:13:14.713 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (42 ms) 2024-01-28T13:13:14.713 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-01-28T13:13:14.728 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (15 ms) 2024-01-28T13:13:14.728 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-01-28T13:13:15.065 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (337 ms) 2024-01-28T13:13:15.065 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-01-28T13:13:16.707 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140213856 notify_id 2993592205315 notifier_gid 15526 2024-01-28T13:13:16.733 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (1668 ms) 2024-01-28T13:13:16.733 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-01-28T13:13:16.899 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140213856 notify_id 2993592205314 notifier_gid 15526 2024-01-28T13:13:16.905 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (172 ms) 2024-01-28T13:13:16.905 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-01-28T13:13:19.516 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140213856 notify_id 2993592205316 notifier_gid 15526 2024-01-28T13:13:19.543 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (2638 ms) 2024-01-28T13:13:19.544 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-01-28T13:13:19.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~160s 2024-01-28T13:13:20.127 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140213856 notify_id 2993592205315 notifier_gid 15526 2024-01-28T13:13:20.136 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (593 ms) 2024-01-28T13:13:20.136 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-01-28T13:13:20.157 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140197760 notify_id 2993592205318 notifier_gid 15526 2024-01-28T13:13:22.563 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (2427 ms) 2024-01-28T13:13:22.563 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-01-28T13:13:22.599 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140255296 notify_id 2993592205319 notifier_gid 15526 2024-01-28T13:13:22.609 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (46 ms) 2024-01-28T13:13:22.609 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-01-28T13:13:22.629 INFO:tasks.workunit.client.0.smithi031.stdout: trying... 2024-01-28T13:13:22.632 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140251120 notify_id 2993592205316 notifier_gid 15526 2024-01-28T13:13:23.632 INFO:tasks.workunit.client.0.smithi031.stdout: timed out 2024-01-28T13:13:23.640 INFO:tasks.workunit.client.0.smithi031.stdout: flushing 2024-01-28T13:13:25.064 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~165s 2024-01-28T13:13:25.632 INFO:tasks.workunit.client.0.smithi031.stdout: flushed 2024-01-28T13:13:25.633 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3023 ms) 2024-01-28T13:13:25.633 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-01-28T13:13:25.649 INFO:tasks.workunit.client.0.smithi031.stdout: trying... 2024-01-28T13:13:25.652 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140251120 notify_id 2993592205317 notifier_gid 15526 2024-01-28T13:13:26.652 INFO:tasks.workunit.client.0.smithi031.stdout: timed out 2024-01-28T13:13:26.656 INFO:tasks.workunit.client.0.smithi031.stdout: flushing 2024-01-28T13:13:28.652 INFO:tasks.workunit.client.0.smithi031.stdout: flushed 2024-01-28T13:13:28.652 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3020 ms) 2024-01-28T13:13:28.653 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-01-28T13:13:28.672 INFO:tasks.workunit.client.0.smithi031.stdout:List watches 2024-01-28T13:13:28.672 INFO:tasks.workunit.client.0.smithi031.stdout:notify2 2024-01-28T13:13:28.674 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140254368 notify_id 2993592205317 notifier_gid 15526 2024-01-28T13:13:30.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~171s 2024-01-28T13:13:31.679 INFO:tasks.workunit.client.0.smithi031.stdout:notify2 done 2024-01-28T13:13:31.679 INFO:tasks.workunit.client.0.smithi031.stdout:watch_check 2024-01-28T13:13:31.679 INFO:tasks.workunit.client.0.smithi031.stdout:unwatch2 2024-01-28T13:13:31.688 INFO:tasks.workunit.client.0.smithi031.stdout: flushing 2024-01-28T13:13:31.688 INFO:tasks.workunit.client.0.smithi031.stdout:done 2024-01-28T13:13:31.688 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3036 ms) 2024-01-28T13:13:31.688 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-01-28T13:13:31.703 INFO:tasks.workunit.client.0.smithi031.stdout:List watches 2024-01-28T13:13:31.703 INFO:tasks.workunit.client.0.smithi031.stdout:notify2 2024-01-28T13:13:31.707 INFO:tasks.workunit.client.0.smithi031.stdout:handle_notify cookie 93907140254368 notify_id 2993592205320 notifier_gid 15526 2024-01-28T13:13:34.711 INFO:tasks.workunit.client.0.smithi031.stdout:notify2 done 2024-01-28T13:13:34.711 INFO:tasks.workunit.client.0.smithi031.stdout:watch_check 2024-01-28T13:13:34.712 INFO:tasks.workunit.client.0.smithi031.stdout:unwatch2 2024-01-28T13:13:34.719 INFO:tasks.workunit.client.0.smithi031.stdout: flushing 2024-01-28T13:13:34.720 INFO:tasks.workunit.client.0.smithi031.stdout:done 2024-01-28T13:13:34.720 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3030 ms) 2024-01-28T13:13:35.098 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (35622 ms total) 2024-01-28T13:13:35.098 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:13:35.098 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:13:35.098 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 16 tests from 2 test suites ran. (37987 ms total) 2024-01-28T13:13:35.099 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 14 tests. 2024-01-28T13:13:35.099 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] 2 tests, listed below: 2024-01-28T13:13:35.099 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify 2024-01-28T13:13:35.099 INFO:tasks.workunit.client.0.smithi031.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-01-28T13:13:35.100 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:13:35.100 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:13:35.100 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_cmd 2024-01-28T13:13:35.120 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:13:35.120 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 4 tests from 1 test suite. 2024-01-28T13:13:35.120 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:13:35.120 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosCmd 2024-01-28T13:13:35.120 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosCmd.MonDescribe 2024-01-28T13:13:35.167 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosCmd.MonDescribe (47 ms) 2024-01-28T13:13:35.167 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosCmd.OSDCmd 2024-01-28T13:13:35.205 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosCmd.OSDCmd (38 ms) 2024-01-28T13:13:35.205 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosCmd.PGCmd 2024-01-28T13:13:36.073 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~176s 2024-01-28T13:13:41.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~182s 2024-01-28T13:13:47.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~187s 2024-01-28T13:13:52.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~193s 2024-01-28T13:13:58.092 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~198s 2024-01-28T13:13:58.106 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosCmd.PGCmd (22901 ms) 2024-01-28T13:13:58.106 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosCmd.WatchLog 2024-01-28T13:13:59.108 INFO:tasks.workunit.client.0.smithi031.stdout:got: 2024-01-28T13:13:58.136482+0000 client.admin [INF] onexx 2024-01-28T13:13:59.108 INFO:tasks.workunit.client.0.smithi031.stdout:got: 2024-01-28T13:13:58.136614+0000 mon.a [INF] from='client.? 172.21.15.31:0/2807728626' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-01-28T13:14:03.109 INFO:tasks.workunit.client.0.smithi031.stdout:got: 2024-01-28T13:14:01.119869+0000 mon.a [INF] from='client.? 172.21.15.31:0/2807728626' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-01-28T13:14:03.160 INFO:tasks.workunit.client.0.smithi031.stdout:got: 2024-01-28T13:14:03.120371+0000 client.admin [INF] threexx 2024-01-28T13:14:03.161 INFO:tasks.workunit.client.0.smithi031.stdout:got: 2024-01-28T13:14:03.120868+0000 mon.a [INF] from='client.? 172.21.15.31:0/2807728626' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-01-28T13:14:03.597 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~204s 2024-01-28T13:14:07.167 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosCmd.WatchLog (9061 ms) 2024-01-28T13:14:07.167 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosCmd (32047 ms total) 2024-01-28T13:14:07.167 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:07.167 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:14:07.167 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 4 tests from 1 test suite ran. (32047 ms total) 2024-01-28T13:14:07.167 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 4 tests. 2024-01-28T13:14:07.168 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:14:07.169 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:14:07.169 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_cmd_pp 2024-01-28T13:14:07.190 INFO:tasks.workunit.client.0.smithi031.stdout:Running main() from gmock_main.cc 2024-01-28T13:14:07.190 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 3 tests from 1 test suite. 2024-01-28T13:14:07.190 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:14:07.190 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosCmd 2024-01-28T13:14:07.190 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosCmd.MonDescribePP 2024-01-28T13:14:07.217 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosCmd.MonDescribePP (28 ms) 2024-01-28T13:14:07.218 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosCmd.OSDCmdPP 2024-01-28T13:14:07.244 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosCmd.OSDCmdPP (26 ms) 2024-01-28T13:14:07.244 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosCmd.PGCmdPP 2024-01-28T13:14:09.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~209s 2024-01-28T13:14:14.606 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~215s 2024-01-28T13:14:20.111 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~220s 2024-01-28T13:14:25.616 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~226s 2024-01-28T13:14:31.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~231s 2024-01-28T13:14:36.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~237s 2024-01-28T13:14:42.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~242s 2024-01-28T13:14:44.112 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosCmd.PGCmdPP (36868 ms) 2024-01-28T13:14:44.112 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 3 tests from LibRadosCmd (36922 ms total) 2024-01-28T13:14:44.112 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:44.112 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:14:44.112 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 3 tests from 1 test suite ran. (36922 ms total) 2024-01-28T13:14:44.112 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 3 tests. 2024-01-28T13:14:44.113 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:14:44.113 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:14:44.113 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_service 2024-01-28T13:14:44.144 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:14:44.154+0000 7f85e82537c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:14:44.144 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:14:44.155+0000 7f85e82537c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:14:44.144 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:14:44.155+0000 7f85e82537c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:14:44.144 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 4 tests from 1 test suite. 2024-01-28T13:14:44.144 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:14:44.145 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosService 2024-01-28T13:14:44.145 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosService.RegisterEarly 2024-01-28T13:14:47.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~248s 2024-01-28T13:14:49.178 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosService.RegisterEarly (5034 ms) 2024-01-28T13:14:49.178 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosService.RegisterLate 2024-01-28T13:14:49.212 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosService.RegisterLate (34 ms) 2024-01-28T13:14:49.213 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosService.StatusFormat 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: cluster: 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: id: 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: health: HEALTH_WARN 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 osds down 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: Degraded data redundancy: 1/6 objects degraded (16.667%), 1 pg degraded, 17 pgs undersized 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: 1 pool(s) do not have an application enabled 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: services: 2024-01-28T13:14:49.340 INFO:tasks.workunit.client.0.smithi031.stdout: mon: 1 daemons, quorum a (age 18m) 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: mgr: x(active, since 18m) 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: osd: 4 osds: 3 up (since 3m), 4 in (since 18m) 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: laundry: 1 daemon active (1 hosts) 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: data: 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: pools: 3 pools, 41 pgs 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: objects: 3 objects, 577 KiB 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: usage: 252 MiB used, 400 GiB / 400 GiB avail 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: pgs: 1/6 objects degraded (16.667%) 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: 24 active+clean 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: 16 active+undersized 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: 1 active+undersized+degraded 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:49.341 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:51.362 INFO:tasks.workunit.client.0.smithi031.stdout: cluster: 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: id: 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: health: HEALTH_WARN 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 osds down 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: Degraded data redundancy: 1/6 objects degraded (16.667%), 1 pg degraded, 17 pgs undersized 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: 1 pool(s) do not have an application enabled 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: services: 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: mon: 1 daemons, quorum a (age 18m) 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: mgr: x(active, since 18m) 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: osd: 4 osds: 3 up (since 3m), 4 in (since 18m) 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: laundry: 1 daemon active (1 hosts) 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: data: 2024-01-28T13:14:51.363 INFO:tasks.workunit.client.0.smithi031.stdout: pools: 3 pools, 41 pgs 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: objects: 3 objects, 577 KiB 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: usage: 252 MiB used, 400 GiB / 400 GiB avail 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: pgs: 1/6 objects degraded (16.667%) 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: 24 active+clean 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: 16 active+undersized 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: 1 active+undersized+degraded 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:51.364 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:53.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~253s 2024-01-28T13:14:53.385 INFO:tasks.workunit.client.0.smithi031.stdout: cluster: 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: id: 68a362e5-2887-44cb-94c6-4a7b059316bf 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: health: HEALTH_WARN 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: 1 osds down 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: Degraded data redundancy: 1/6 objects degraded (16.667%), 1 pg degraded, 17 pgs undersized 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: 1 pool(s) do not have an application enabled 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: services: 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: mon: 1 daemons, quorum a (age 18m) 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: mgr: x(active, since 18m) 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: osd: 4 osds: 3 up (since 3m), 4 in (since 18m) 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: foo: 16 portals active (1 hosts, 3 zones) 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: data: 2024-01-28T13:14:53.386 INFO:tasks.workunit.client.0.smithi031.stdout: pools: 3 pools, 41 pgs 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: objects: 3 objects, 577 KiB 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: usage: 252 MiB used, 400 GiB / 400 GiB avail 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: pgs: 1/6 objects degraded (16.667%) 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: 24 active+clean 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: 16 active+undersized 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: 1 active+undersized+degraded 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:53.387 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:14:53.409 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosService.StatusFormat (4197 ms) 2024-01-28T13:14:53.409 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosService.Status 2024-01-28T13:14:58.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~259s 2024-01-28T13:15:04.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~264s 2024-01-28T13:15:09.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~270s 2024-01-28T13:15:13.446 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosService.Status (20036 ms) 2024-01-28T13:15:13.446 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosService (29303 ms total) 2024-01-28T13:15:13.447 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:15:13.447 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:15:13.447 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 4 tests from 1 test suite ran. (29303 ms total) 2024-01-28T13:15:13.447 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 4 tests. 2024-01-28T13:15:13.452 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-01-28T13:15:13.452 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -eq 1 ']' 2024-01-28T13:15:13.452 INFO:tasks.workunit.client.0.smithi031.stderr:+ ceph_test_rados_api_service_pp 2024-01-28T13:15:13.483 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:15:13.493+0000 7f584344f7c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:15:13.483 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:15:13.494+0000 7f584344f7c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:15:13.483 INFO:tasks.workunit.client.0.smithi031.stderr:2024-01-28T13:15:13.494+0000 7f584344f7c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:15:13.483 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] Running 4 tests from 1 test suite. 2024-01-28T13:15:13.483 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment set-up. 2024-01-28T13:15:13.483 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosServicePP 2024-01-28T13:15:13.483 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosServicePP.RegisterEarly 2024-01-28T13:15:15.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~275s 2024-01-28T13:15:18.516 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosServicePP.RegisterEarly (5033 ms) 2024-01-28T13:15:18.516 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosServicePP.RegisterLate 2024-01-28T13:15:18.564 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosServicePP.RegisterLate (48 ms) 2024-01-28T13:15:18.564 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosServicePP.Status 2024-01-28T13:15:20.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~281s 2024-01-28T13:15:26.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~286s 2024-01-28T13:15:31.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~292s 2024-01-28T13:15:37.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~297s 2024-01-28T13:15:38.602 INFO:tasks.workunit.client.0.smithi031.stdout:[ OK ] LibRadosServicePP.Status (20038 ms) 2024-01-28T13:15:38.603 INFO:tasks.workunit.client.0.smithi031.stdout:[ RUN ] LibRadosServicePP.Close 2024-01-28T13:15:38.603 INFO:tasks.workunit.client.0.smithi031.stdout:attempt 0 of 20 2024-01-28T13:15:42.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~303s 2024-01-28T13:15:42.686 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-01-28T13:15:43.189 INFO:tasks.ceph.osd.0:Sent signal 15 2024-01-28T13:15:43.190 INFO:tasks.ceph.osd.1:Sent signal 15 2024-01-28T13:15:43.190 INFO:tasks.ceph.osd.3:Sent signal 15 2024-01-28T13:15:43.191 INFO:tasks.ceph.mon.a:Sent signal 15 2024-01-28T13:15:43.191 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-01-28T13:15:43.192 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-01-28T13:15:43.204+0000 7f3fd4df7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 29749) UID: 0 2024-01-28T13:15:43.231 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-28T13:15:43.243+0000 7fd26ebfc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 30096) UID: 0 2024-01-28T13:15:43.232 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-01-28T13:15:43.243+0000 7fd26ebfc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-01-28T13:20:43.653 INFO:tasks.workunit.client.0.smithi031.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-933-g00c0d54d/rpm/el9/BUILD/ceph-19.0.0-933-g00c0d54d/src/test/librados/service_cxx.cc:90: Failure 2024-01-28T13:20:43.653 INFO:tasks.workunit.client.0.smithi031.stdout:Expected equality of these values: 2024-01-28T13:20:43.653 INFO:tasks.workunit.client.0.smithi031.stdout: 0 2024-01-28T13:20:43.654 INFO:tasks.workunit.client.0.smithi031.stdout: cluster.connect() 2024-01-28T13:20:43.654 INFO:tasks.workunit.client.0.smithi031.stdout: Which is: -110 2024-01-28T13:20:43.654 INFO:tasks.workunit.client.0.smithi031.stdout:[ FAILED ] LibRadosServicePP.Close (305052 ms) 2024-01-28T13:20:43.654 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] 4 tests from LibRadosServicePP (330172 ms total) 2024-01-28T13:20:43.654 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:20:43.654 INFO:tasks.workunit.client.0.smithi031.stdout:[----------] Global test environment tear-down 2024-01-28T13:20:43.655 INFO:tasks.workunit.client.0.smithi031.stdout:[==========] 4 tests from 1 test suite ran. (330172 ms total) 2024-01-28T13:20:43.655 INFO:tasks.workunit.client.0.smithi031.stdout:[ PASSED ] 3 tests. 2024-01-28T13:20:43.655 INFO:tasks.workunit.client.0.smithi031.stdout:[ FAILED ] 1 test, listed below: 2024-01-28T13:20:43.655 INFO:tasks.workunit.client.0.smithi031.stdout:[ FAILED ] LibRadosServicePP.Close 2024-01-28T13:20:43.655 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-01-28T13:20:43.655 INFO:tasks.workunit.client.0.smithi031.stdout: 1 FAILED TEST 2024-01-28T13:20:43.656 INFO:tasks.workunit.client.0.smithi031.stderr:++ cleanup 2024-01-28T13:20:43.656 INFO:tasks.workunit.client.0.smithi031.stderr:++ pkill -P 32088 2024-01-28T13:20:43.670 INFO:tasks.workunit.client.0.smithi031.stderr:++ true 2024-01-28T13:20:43.671 INFO:tasks.workunit.client.0.smithi031.stderr:+ cleanup 2024-01-28T13:20:43.671 INFO:tasks.workunit.client.0.smithi031.stderr:+ pkill -P 32088 2024-01-28T13:20:43.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T13:20:43.675 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2024-01-28T13:20:43.676 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-01-28T13:20:43.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-01-28T13:20:43.981 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7eb9e33f53ab64f23bcf0a2a31509f3442110e4e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-01-28T13:20:45.685 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5ceab8787d024829aee00496e08e71a2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7eb9e33f53ab64f23bcf0a2a31509f3442110e4e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-01-28T13:20:45.687 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-28T13:20:45.697 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-28T13:20:45.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-28T13:20:45.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.841+0000 7f111d304640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-01-28T13:20:45.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f111d304640 1 Processor -- start 2024-01-28T13:20:45.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f111d304640 1 -- start start 2024-01-28T13:20:45.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f111d304640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f1118136760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:20:45.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f111d304640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:20:45.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f111d304640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f11180709f0 2024-01-28T13:20:45.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f11180702e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:45.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f1118136760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:45.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:45.846+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f1118136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:20:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:46.047+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f11180702e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:46.047+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f1118136760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:46.047+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f1118136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:20:46.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:46.448+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f11180702e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:46.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:46.448+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f1118136760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:46.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:46.448+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f1118136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:20:47.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:47.249+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f11180702e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:47.249+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f1118136760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:47.249+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f1118136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:20:48.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f1118136760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f1118136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:20:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f11180702e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:20:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:20:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f11180709f0 2024-01-28T13:20:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:48.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:48.847+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:20:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:49.048+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:49.048+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:49.048+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:20:49.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:49.449+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:49.449+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:49.449+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:20:50.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:50.250+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:50.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:50.250+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:50.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:50.250+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:20:51.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:51.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:20:51.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:51.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:20:51.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:20:51.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f11180709f0 2024-01-28T13:20:51.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:51.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:51.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:51.847+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:20:52.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:52.048+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:52.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:52.048+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:52.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:52.048+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:20:52.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:52.449+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:52.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:52.449+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:52.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:52.449+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:53.250+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:53.250+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:53.250+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:20:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:20:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:20:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:20:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f11180709f0 2024-01-28T13:20:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:54.847+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:20:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:55.048+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:55.048+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:55.048+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:20:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:55.449+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:55.449+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:55.449+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:20:56.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:56.250+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:56.250+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:56.250+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:20:57.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.847+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:57.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.847+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:20:57.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.847+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:20:57.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.848+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:20:57.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.848+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:20:57.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.848+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f11180709f0 2024-01-28T13:20:57.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.848+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:57.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.848+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:57.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:57.848+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:20:58.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:58.049+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:58.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:58.049+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:58.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:58.049+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:20:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:58.450+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:58.450+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:58.450+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:20:59.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:59.251+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:20:59.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:59.251+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:20:59.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:20:59.251+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:00.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:00.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11180709f0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:00.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:00.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:00.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:00.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009540 2024-01-28T13:21:00.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:00.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:00.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:00.848+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:01.049+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:01.049+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:01.049+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:01.450+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:01.450+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:01.450+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:02.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:02.251+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:02.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:02.251+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:02.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:02.251+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:21:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009540 2024-01-28T13:21:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:03.848+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:04.049+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:04.049+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:04.049+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:04.450+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:04.450+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:04.450+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:05.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:05.251+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:05.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:05.251+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:05.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:05.251+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009540 2024-01-28T13:21:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.848+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:06.849+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:07.050+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:07.050+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:07.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:07.050+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:07.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:07.451+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:07.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:07.451+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:07.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:07.451+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:08.252+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:08.252+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:08.252+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:09.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009540 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:09.849+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:10.050+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:10.050+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:10.050+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:10.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:10.451+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:10.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:10.451+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:10.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:10.451+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:11.252+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:11.252+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:11.252+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009540 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8007bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f800c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:12.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:12.849+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f800c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:13.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:13.050+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:13.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:13.050+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:13.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:13.050+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f800c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:13.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:13.451+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:13.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:13.451+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:13.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:13.451+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f800c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:14.252+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:14.252+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:14.252+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f800c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:15.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:15.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f800c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:15.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:21:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:15.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:15.849+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:16.050+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:16.050+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:16.050+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:16.451+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:16.451+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:16.451+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:17.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:17.252+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:17.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:17.252+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:17.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:17.252+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:18.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:18.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:18.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:18.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:18.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:18.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:18.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:18.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:18.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:18.850+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:19.051+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:19.051+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:19.051+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:19.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:19.452+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:19.452+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:19.452+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:20.253+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:20.253+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:20.253+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:21.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:21.850+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:22.051+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:22.051+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:22.051+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:22.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:22.452+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:22.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:22.452+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:22.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:22.452+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:23.253+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:23.253+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:23.253+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:24.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:24.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:24.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:24.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:24.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:24.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:24.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:24.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:24.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:24.850+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:25.051+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:25.051+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:25.051+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:25.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:25.452+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:25.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:25.452+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:25.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:25.452+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:26.253+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:26.253+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:26.253+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:27.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:27.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:27.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:21:27.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:27.851+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:28.052+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:28.052+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:28.052+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:28.453+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:28.453+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:28.453+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:29.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:29.254+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:29.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:29.254+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:29.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:29.254+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:30.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:30.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:30.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:30.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:30.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:30.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:30.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:30.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:30.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:30.851+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:31.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:31.052+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:31.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:31.052+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:31.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:31.052+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:31.453+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:31.453+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:31.453+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:32.254+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:32.254+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:32.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:32.254+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:21:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:33.851+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:34.052+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:34.052+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:34.052+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:34.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:34.453+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:34.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:34.453+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:34.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:34.453+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:35.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:35.254+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:35.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:35.254+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:35.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:35.254+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:36.852+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:37.053+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:37.053+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:37.053+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:37.454+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:37.454+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:37.454+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:38.255+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:38.255+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:38.255+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:39.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:39.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:39.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:39.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:39.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f111806fed0 2024-01-28T13:21:39.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:39.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:39.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:39.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:39.852+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:40.053+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:40.053+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:40.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:40.053+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:40.454+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:40.454+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:40.454+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:41.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:41.255+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:41.255+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:41.255+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f111806fed0 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:42.852+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:43.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:43.053+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:43.053+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:43.053+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:43.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:43.454+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:43.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:43.454+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:43.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:43.454+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:44.255+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:44.255+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:44.255+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:45.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:45.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:45.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f111806fed0 legacy=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:45.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8008340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:45.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f8008a50 2024-01-28T13:21:45.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:45.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008a50 legacy=0x7f10f8006bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:45.853+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:46.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:46.054+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008a50 legacy=0x7f10f8006bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:46.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:46.054+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:46.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:46.054+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:46.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:46.455+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:46.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:46.455+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008a50 legacy=0x7f10f8006bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:46.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:46.455+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:47.256+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:47.256+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008a50 legacy=0x7f10f8006bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:47.256+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f8008340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f8008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008a50 legacy=0x7f10f8006bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f8008e60 2024-01-28T13:21:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008e60 legacy=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:48.853+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:49.054+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:49.054+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008e60 legacy=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:49.054+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:49.455+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008e60 legacy=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:49.455+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:49.455+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:50.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:50.256+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008e60 legacy=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:50.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:50.256+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:50.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:50.256+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:51.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:51.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:51.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008e60 legacy=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:51.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:51.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f8008320 2024-01-28T13:21:51.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f8009ab0 2024-01-28T13:21:51.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:51.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:51.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:51.854+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:52.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:52.055+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:52.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:52.055+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:52.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:52.055+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:52.456+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:52.456+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:52.456+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:53.257+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:53.257+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:53.257+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:54.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 msgr2=0x7f10f80055b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f8009ab0 0x7f10f80055b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800c540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f8008320 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f80035a0 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:54.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:54.853+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:55.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:55.054+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:55.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:55.054+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:55.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:55.054+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:55.455+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:55.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:55.455+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:55.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:55.455+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:56.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:56.256+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:56.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:56.256+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:56.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:56.256+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:21:57.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.853+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:57.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:21:57.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.853+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:21:57.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.853+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:21:57.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.853+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f8008320 2024-01-28T13:21:57.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.854+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f80035a0 2024-01-28T13:21:57.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.854+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:57.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.854+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:21:57.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:57.854+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:58.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:58.055+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:58.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:58.055+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:58.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:58.055+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:21:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:58.456+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:58.456+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:58.456+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:21:59.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:59.257+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:21:59.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:59.257+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:21:59.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:21:59.257+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:00.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f8005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:00.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:00.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f8008320 legacy=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:00.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f800a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:00.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800bc50 2024-01-28T13:22:00.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f80035a0 2024-01-28T13:22:00.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800bc50 legacy=0x7f10f800e070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:00.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:00.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:00.854+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f800a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:01.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:01.055+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800bc50 legacy=0x7f10f800e070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:01.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:01.055+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:01.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:01.055+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f800a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:01.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:01.456+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800bc50 legacy=0x7f10f800e070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:01.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:01.456+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:01.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:01.456+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f800a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:02.257+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800bc50 legacy=0x7f10f800e070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:02.257+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:02.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:02.257+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f800a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:03.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 msgr2=0x7f10f800a530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:03.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f80035a0 0x7f10f800a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:03.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800bc50 legacy=0x7f10f800e070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:03.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 0x7f10f8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:03.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f800a530 2024-01-28T13:22:03.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800aa70 2024-01-28T13:22:03.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:03.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:03.854+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:04.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:04.055+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:04.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:04.055+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:04.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:04.055+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:04.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:04.456+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:04.456+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:04.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:04.456+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:05.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:05.257+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:05.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:05.257+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:05.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:05.257+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800aa70 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f800c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800a530 2024-01-28T13:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800b590 2024-01-28T13:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:06.854+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:07.055+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:07.055+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:07.055+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:07.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:07.456+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:07.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:07.456+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:07.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:07.456+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:08.257+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:08.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:08.257+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:08.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:08.257+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:09.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:09.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:09.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:09.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f800a530 2024-01-28T13:22:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800b590 2024-01-28T13:22:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:09.855+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:10.056+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:10.056+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:10.056+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:10.457+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:10.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:10.457+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:10.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:10.457+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:11.258+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:11.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:11.258+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:11.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:11.258+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800b590 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800a530 2024-01-28T13:22:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:12.855+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:13.056+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:13.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:13.056+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:13.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:13.056+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:13.457+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:13.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:13.457+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:13.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:13.457+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:14.258+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:14.258+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:14.258+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:15.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:15.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f800a530 2024-01-28T13:22:15.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:15.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:15.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:15.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:15.855+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:16.056+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:16.056+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:16.056+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:16.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:16.457+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:16.457+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:16.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:16.457+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:17.258+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:17.258+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:17.258+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:18.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800b590 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:18.855+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:19.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:19.056+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:19.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:19.056+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:19.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:19.056+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:19.457+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:19.457+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:19.457+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:20.258+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:20.258+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:20.258+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f800a530 2024-01-28T13:22:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.855+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:21.856+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:22.057+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:22.057+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:22.057+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:22.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:22.458+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:22.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:22.458+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:22.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:22.458+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:23.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:23.259+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:23.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:23.259+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:23.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:23.259+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800b590 2024-01-28T13:22:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:24.856+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800ae80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:25.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:25.057+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:25.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:25.057+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:25.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:25.057+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800ae80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:25.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:25.458+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:25.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:25.458+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:25.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:25.458+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800ae80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:26.259+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:26.259+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:26.259+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800ae80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:27.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:27.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800ae80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800b590 legacy=0x7f10f800f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f800a530 2024-01-28T13:22:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:27.856+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:28.057+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:28.057+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:28.057+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:28.458+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:28.458+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:28.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:28.458+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:29.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:29.259+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:29.259+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:29.259+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:30.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800c770 2024-01-28T13:22:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800c770 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:30.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.856+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:30.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:30.857+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:31.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:31.057+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800c770 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:31.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:31.058+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:31.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:31.058+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:31.458+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800c770 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:31.459+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:31.459+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:32.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:32.259+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800c770 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:32.260+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:32.260+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800c770 legacy=0x7f10f800ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f800a530 2024-01-28T13:22:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f80104a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:33.857+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:34.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:34.058+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f80104a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:34.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:34.058+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:34.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:34.058+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:34.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:34.459+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f80104a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:34.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:34.459+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:34.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:34.459+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:35.260+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f80104a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:35.260+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:35.260+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f8007bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f8007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f80104a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800ae80 2024-01-28T13:22:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800ae80 legacy=0x7f10f800b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:36.857+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:37.058+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:37.058+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800ae80 legacy=0x7f10f800b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:37.058+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:37.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:37.459+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:37.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:37.459+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800ae80 legacy=0x7f10f800b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:37.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:37.459+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:38.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:38.260+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:38.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:38.260+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800ae80 legacy=0x7f10f800b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:38.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:38.260+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800ae80 legacy=0x7f10f800b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180983e0 con 0x7f10f800a530 2024-01-28T13:22:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:39.858+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:40.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:40.059+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:40.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:40.059+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:40.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:40.059+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:40.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:40.460+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:40.460+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:40.460+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:41.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:41.261+0000 7f11167fc640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:41.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:41.261+0000 7f1116ffd640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:41.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:41.261+0000 7f11167fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1114ff9640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-28T13:22:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1114ff9640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-28T13:22:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1114ff9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-28T13:22:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1114ff9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1118098940 con 0x7f10f800a530 2024-01-28T13:22:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1114ff9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118072c30 con 0x7f10f800d070 2024-01-28T13:22:42.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:42.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:42.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:42.858+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-28T13:22:43.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:43.059+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:43.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:43.059+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:43.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:43.059+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-28T13:22:43.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:43.460+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:43.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:43.460+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:43.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:43.460+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-28T13:22:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:44.261+0000 7f1116ffd640 1 -- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 msgr2=0x7f10f800c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.31:3300/0 2024-01-28T13:22:44.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:44.261+0000 7f11167fc640 1 -- >> v1:172.21.15.31:6789/0 conn(0x7f10f800a530 legacy=0x7f10f800c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.31:6789/0 2024-01-28T13:22:44.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-01-28T13:22:44.261+0000 7f1116ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f800d070 0x7f10f800c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-28T13:22:45.747 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-01-28T13:22:45.748 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-01-28T13:22:45.759 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-28T13:22:45.759 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-28T13:22:45.759 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-28T13:22:45.759 INFO:teuthology.orchestra.run:waiting for 300 2024-01-28T13:22:45.856 INFO:tasks.ceph.osd.0:Stopped 2024-01-28T13:22:45.856 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-28T13:22:45.856 INFO:teuthology.orchestra.run:waiting for 300 2024-01-28T13:22:51.957 INFO:tasks.ceph.osd.1:Stopped 2024-01-28T13:22:51.958 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-28T13:22:51.958 INFO:teuthology.orchestra.run:waiting for 300 2024-01-28T13:22:51.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T13:22:51.958 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi137 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log' 2024-01-28T13:22:51.969 INFO:tasks.ceph.osd.2:Stopped 2024-01-28T13:22:51.969 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-28T13:22:51.969 INFO:teuthology.orchestra.run:waiting for 300 2024-01-28T13:22:52.068 INFO:tasks.ceph.osd.3:Stopped 2024-01-28T13:22:52.068 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-28T13:22:52.068 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-28T13:22:52.068 INFO:teuthology.orchestra.run:waiting for 300 2024-01-28T13:22:52.068 INFO:tasks.ceph.mgr.x:Stopped 2024-01-28T13:22:52.069 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-28T13:22:52.069 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-28T13:22:52.069 INFO:teuthology.orchestra.run:waiting for 300 2024-01-28T13:22:52.069 INFO:tasks.ceph.mon.a:Stopped 2024-01-28T13:22:52.069 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-28T13:22:52.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-01-28T13:22:52.108 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-28T13:22:52.108 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-28T13:22:52.250 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-01-28T13:22:52.250 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-28T13:22:52.323 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-28T13:22:52.323 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-28T13:22:52.399 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi137.front.sepia.ceph.com 2024-01-28T13:22:52.400 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-28T13:22:52.473 INFO:tasks.ceph:Archiving mon data... 2024-01-28T13:22:52.475 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/data/mon.a.tgz 2024-01-28T13:22:52.475 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-01-28T13:22:52.493 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.PMQUmC2aTw 2024-01-28T13:22:52.494 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PMQUmC2aTw 2024-01-28T13:22:52.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.PMQUmC2aTw 2024-01-28T13:22:52.723 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.PMQUmC2aTw is 291KB 2024-01-28T13:22:52.817 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.PMQUmC2aTw 2024-01-28T13:22:52.832 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-28T13:22:52.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-28T13:22:52.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-28T13:22:52.953 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-28T13:22:52.970 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-28T13:22:52.970 INFO:tasks.ceph:Archiving crash dumps... 2024-01-28T13:22:52.972 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/remote/smithi031/crash 2024-01-28T13:22:52.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-28T13:22:53.021 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/remote/smithi137/crash 2024-01-28T13:22:53.022 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-28T13:22:53.082 INFO:tasks.ceph:Compressing logs... 2024-01-28T13:22:53.082 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T13:22:53.085 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-28T13:22:53.109 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33399.log 2024-01-28T13:22:53.109 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-28T13:22:53.109 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.33399.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31829.log 2024-01-28T13:22:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29829.log 2024-01-28T13:22:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29949.log 2024-01-28T13:22:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.29829.log: /var/log/ceph/ceph-client.admin.29949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29909.log 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.29829.log.gz 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.3.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.29949.log.gz 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.29909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29869.log 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.29909.log.gz 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-28T13:22:53.121 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.29869.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.29869.log.gz 2024-01-28T13:22:53.137 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32558.log 2024-01-28T13:22:53.137 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31241.log 2024-01-28T13:22:53.137 INFO:teuthology.orchestra.run.smithi031.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33399.log.gz 2024-01-28T13:22:53.137 INFO:teuthology.orchestra.run.smithi031.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.31829.log.gz 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95253.log 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.32558.log: /var/log/ceph/ceph-client.admin.31241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30361.log 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95253.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.31241.log.gz 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95771.log 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31677.log 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.30361.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95253.log.gz 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33534.log 2024-01-28T13:22:53.138 INFO:teuthology.orchestra.run.smithi031.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.30361.log.gz 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30333.log 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.31677.log.gz 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95626.log 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.30333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95665.log 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.30333.log.gz/var/log/ceph/ceph-client.admin.95626.log: 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.95771.log.gz 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-28T13:22:53.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31678.log 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95626.log.gz 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95704.log 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31177.log 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95704.log: 90.9%/var/log/ceph/ceph-client.admin.31678.log: -- replaced with /var/log/ceph/ceph-client.admin.95665.log.gz 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.31678.log.gz 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33362.log 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-28T13:22:53.140 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31367.log 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31177.log.gz 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32089.log 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1%/var/log/ceph/ceph-client.admin.31367.log: -- replaced with /var/log/ceph/ceph-client.admin.95704.log.gz 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33457.log 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31367.log.gz 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33362.log.gz 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95685.log 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.32089.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-28T13:22:53.141 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33457.log: /var/log/ceph/ceph-client.admin.95685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33495.log 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.1.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.95685.log.gz 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96285.log 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33495.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33495.log.gz 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96173.log 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.96285.log: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.96285.log.gz 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95646.log 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.96173.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.32089.log.gz 2024-01-28T13:22:53.142 INFO:teuthology.orchestra.run.smithi031.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.32558.log.gz 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31509.log 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31675.log 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31509.log: /var/log/ceph/ceph-client.admin.95646.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.31509.log.gz 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95646.log.gz 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30177.log 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.33457.log.gz 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95822.log 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31675.log: /var/log/ceph/ceph-client.admin.30177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31283.log 2024-01-28T13:22:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95822.log: 84.7% 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31675.log.gz 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30177.log.gz 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.96173.log.gz 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31508.log 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31679.log 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31283.log: /var/log/ceph/ceph-client.admin.31508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95472.log 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr: 87.2%/var/log/ceph/ceph-client.admin.31679.log: -- replaced with /var/log/ceph/ceph-client.admin.31508.log.gz 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31283.log.gz 2024-01-28T13:22:53.144 INFO:teuthology.orchestra.run.smithi031.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31679.log.gz 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31510.log 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31325.log 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31409.log 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.31510.log.gz 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31325.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.95472.log.gz 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31325.log.gz 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95291.log 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31409.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-28T13:22:53.145 INFO:teuthology.orchestra.run.smithi031.stderr: 86.0% 91.6% -- replaced with /var/log/ceph/ceph-client.admin.95822.log.gz 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31409.log.gz/var/log/ceph/ceph-client.admin.95291.log: 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33418.log 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.33418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30098.log 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33418.log.gz 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31511.log 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.30098.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.95291.log.gz 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.30098.log.gz 2024-01-28T13:22:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31871.log 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95509.log 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31871.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.31511.log.gz 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95180.log 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.31871.log.gz 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95509.log: 88.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.95509.log.gz 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31787.log 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.95180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33514.log 2024-01-28T13:22:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31787.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27790.log 2024-01-28T13:22:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33514.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.31787.log.gz 2024-01-28T13:22:53.148 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33514.log.gz 2024-01-28T13:22:53.150 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.tmp-client.admin.27790.log: 66.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27790.log.gz 2024-01-28T13:22:53.150 INFO:teuthology.orchestra.run.smithi031.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.95180.log.gz 2024-01-28T13:22:53.320 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.2.log: 92.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-28T13:22:53.621 INFO:teuthology.orchestra.run.smithi031.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33534.log.gz 2024-01-28T13:22:54.759 INFO:teuthology.orchestra.run.smithi031.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-28T13:23:07.898 INFO:teuthology.orchestra.run.smithi137.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-28T13:23:12.472 INFO:teuthology.orchestra.run.smithi137.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-28T13:23:12.475 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-01-28T13:23:12.475 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m19.375s 2024-01-28T13:23:12.475 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m32.994s 2024-01-28T13:23:12.475 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m1.312s 2024-01-28T13:23:13.769 INFO:teuthology.orchestra.run.smithi031.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-28T13:23:15.184 INFO:teuthology.orchestra.run.smithi031.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-28T13:23:15.185 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-01-28T13:23:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m22.090s 2024-01-28T13:23:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m43.293s 2024-01-28T13:23:15.186 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m1.620s 2024-01-28T13:23:15.186 INFO:tasks.ceph:Archiving logs... 2024-01-28T13:23:15.187 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/remote/smithi031/log 2024-01-28T13:23:15.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-28T13:23:21.873 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/remote/smithi137/log 2024-01-28T13:23:21.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-28T13:23:27.208 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-01-28T13:23:27.219 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-28T13:23:27.229 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-01-28T13:23:27.233 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-28T13:23:27.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-28T13:23:27.239 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-28T13:23:27.319 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-28T13:23:27.328 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-28T13:23:27.328 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-28T13:23:27.331 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-28T13:23:27.344 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-01-28T13:23:27.347 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-01-28T13:23:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-28T13:23:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-01-28T13:23:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +714us[ +750us] +/- 79ms 2024-01-28T13:23:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -720us[ -684us] +/- 63ms 2024-01-28T13:23:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +271us[ +307us] +/- 85ms 2024-01-28T13:23:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T13:23:27.351 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-28T13:23:27.352 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-01-28T13:23:27.352 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +707us[ +732us] +/- 79ms 2024-01-28T13:23:27.352 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -713us[ -688us] +/- 63ms 2024-01-28T13:23:27.352 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +268us[ +293us] +/- 85ms 2024-01-28T13:23:27.352 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-28T13:23:27.353 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-28T13:23:27.367 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-28T13:23:27.368 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-28T13:23:27.435 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-28T13:23:27.504 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-28T13:23:27.574 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-28T13:23:27.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T13:23:27.604 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-28T13:23:27.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T13:23:27.637 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-28T13:23:27.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-28T13:23:27.676 INFO:teuthology.task.internal:Duration was 2096.940230 seconds 2024-01-28T13:23:27.676 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-28T13:23:27.718 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-28T13:23:27.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-28T13:23:27.721 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-28T13:23:27.762 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-28T13:23:27.766 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-28T13:23:28.222 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-28T13:23:28.222 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-01-28T13:23:28.223 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-28T13:23:28.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-01-28T13:23:28.244 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-28T13:23:28.272 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-28T13:23:28.273 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-28T13:23:28.287 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-28T13:23:28.343 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-28T13:23:28.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-28T13:23:28.357 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-28T13:23:28.477 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-28T13:23:28.487 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-28T13:23:28.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-28T13:23:28.514 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-28T13:23:28.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-28T13:23:28.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-28T13:23:28.556 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-28T13:23:28.579 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-01-28T13:23:28.606 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-01-28T13:23:28.639 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-28T13:23:28.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T13:23:28.657 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-28T13:23:28.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T13:23:28.693 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-28T13:23:28.703 INFO:teuthology.task.internal:Transferring archived files... 2024-01-28T13:23:28.704 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/remote/smithi031 2024-01-28T13:23:28.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-28T13:23:28.740 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799/remote/smithi137 2024-01-28T13:23:28.740 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-28T13:23:28.775 INFO:teuthology.task.internal:Removing archive directory... 2024-01-28T13:23:28.775 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-28T13:23:28.778 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-28T13:23:28.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-28T13:23:28.837 INFO:teuthology.task.internal:Not uploading archives. 2024-01-28T13:23:28.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-28T13:23:28.869 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-28T13:23:28.869 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-28T13:23:28.871 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-28T13:23:28.883 INFO:teuthology.orchestra.run.smithi031.stdout: 265291 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 13:23 /home/ubuntu/cephtest 2024-01-28T13:23:28.883 INFO:teuthology.orchestra.run.smithi031.stdout: 276951 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 12:56 /home/ubuntu/cephtest/mnt.0 2024-01-28T13:23:28.883 INFO:teuthology.orchestra.run.smithi031.stdout: 276952 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 12:57 /home/ubuntu/cephtest/mnt.0/client.0 2024-01-28T13:23:28.883 INFO:teuthology.orchestra.run.smithi031.stdout: 290994 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 28 12:57 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-01-28T13:23:28.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-28T13:23:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-01-28T13:23:28.885 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_7eb9e33f53ab64f23bcf0a2a31509f3442110e4e/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-01-28T13:23:28.897 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-28T13:23:28.930 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-28T13:23:28.986 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi031 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-01-28T13:23:28.987 INFO:teuthology.nuke:Checking targets against current locks 2024-01-28T13:23:28.995 INFO:teuthology.orchestra.run.smithi137.stdout: 266028 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 28 13:23 /home/ubuntu/cephtest 2024-01-28T13:23:29.018 DEBUG:teuthology.nuke:shortname: smithi031 2024-01-28T13:23:29.019 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-28T13:23:29.044 DEBUG:teuthology.nuke:shortname: smithi137 2024-01-28T13:23:29.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-28T13:23:29.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-28 12:40:23.044797', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+e/mN47WL+k2UShx4+QdYe2PoCv9kFTOWTk6NUp62n3KES5c+ZwdMP9U4M1mNve/sZ1c2dU0QAIgqEU58p7p8='} 2024-01-28T13:23:29.074 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-01-28_12:38:29-crimson-rados-wip-matanb-crimson-pg-map-logs-distro-crimson-smithi/7535799', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-28 12:40:23.046374', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIydCY7OmjmfsVwUzUdkkPVrOiO0zWzLJGxQNS0ZorlF1m7YkqFZ0KHhg6Emxon2MVNwuKOiZXxfyftCEFTDDvA='} 2024-01-28T13:23:29.095 INFO:teuthology.orchestra.console.smithi031:Power off 2024-01-28T13:23:29.095 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-28T13:23:29.120 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2024-01-28T13:23:29.120 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T13:23:29.135 INFO:teuthology.orchestra.console.smithi137:Power off 2024-01-28T13:23:29.135 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-28T13:23:29.143 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-01-28T13:23:29.160 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-01-28T13:23:29.160 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T13:23:29.181 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-01-28T13:23:33.144 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T13:23:33.182 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T13:23:33.258 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-01-28T13:23:33.291 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-01-28T13:23:37.263 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T13:23:37.292 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-28T13:23:37.373 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-01-28T13:23:37.402 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-01-28T13:23:37.475 INFO:teuthology.orchestra.console.smithi031:Power off completed 2024-01-28T13:23:37.503 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-01-28T13:23:37.609 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 2096.9402301311493 failure_reason: 'Command failed (workunit test rados/test.sh) on smithi031 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7eb9e33f53ab64f23bcf0a2a31509f3442110e4e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson''' flavor: crimson owner: scheduled_matan@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5ceab8787d024829aee00496e08e71a2 status: fail success: false 2024-01-28T13:23:37.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-28T13:23:37.684 INFO:teuthology.run:FAIL