2024-04-16T07:33:15.041 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T07:33:15.041 DEBUG:teuthology.run:Teuthology command: teuthology --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} --name matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi --verbose --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979 -- /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/orig.config.yaml 2024-04-16T07:33:15.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T07:33:15.154 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979 branch: wip-matanb-crimson-replicated-april-15 description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: null first_in_suite: false job_id: '7657979' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-replicated-april-15 ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 30b0aa0009a925d6a46536e55c5656edcd51ecdb ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 30b0aa0009a925d6a46536e55c5656edcd51ecdb selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: a7639cb6ce633dabc4d37769d62caacb54582984 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 6444 sha1: 30b0aa0009a925d6a46536e55c5656edcd51ecdb sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: a7639cb6ce633dabc4d37769d62caacb54582984 targets: smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ3frryKjjuLRnl70qaclrLNafzBX76iNXxFw3R1ss5XsTmwjUsliogWn4rfLFlamMR2t9AaHgOzLQU2HkbJd8s= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCAVTSCYMgv1PQYlBnurqYd7NhPHCnsHcIcI6lKQiTWoqpmW7iVCioIXHo5yYKnvin+Ne7wIGNkWxplxHDc58s= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_07:26:08 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-16T07:33:15.154 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa; will attempt to use it 2024-04-16T07:33:15.155 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks 2024-04-16T07:33:15.155 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-16T07:33:15.157 INFO:teuthology.task.internal:Checking packages... 2024-04-16T07:33:15.183 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '30b0aa0009a925d6a46536e55c5656edcd51ecdb' 2024-04-16T07:33:15.183 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-16T07:33:15.183 INFO:teuthology.packaging:ref: None 2024-04-16T07:33:15.184 INFO:teuthology.packaging:tag: None 2024-04-16T07:33:15.184 INFO:teuthology.packaging:branch: wip-matanb-crimson-replicated-april-15 2024-04-16T07:33:15.184 INFO:teuthology.packaging:sha1: 30b0aa0009a925d6a46536e55c5656edcd51ecdb 2024-04-16T07:33:15.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-replicated-april-15 2024-04-16T07:33:15.331 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3109.g30b0aa00 2024-04-16T07:33:15.333 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-16T07:33:15.339 INFO:teuthology.task.internal:no buildpackages task found 2024-04-16T07:33:15.340 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-16T07:33:15.348 INFO:teuthology.task.internal:Saving configuration 2024-04-16T07:33:15.363 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-16T07:33:15.371 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T07:33:15.399 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 07:27:14.632469', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ3frryKjjuLRnl70qaclrLNafzBX76iNXxFw3R1ss5XsTmwjUsliogWn4rfLFlamMR2t9AaHgOzLQU2HkbJd8s='} 2024-04-16T07:33:15.423 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 07:27:14.634229', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCAVTSCYMgv1PQYlBnurqYd7NhPHCnsHcIcI6lKQiTWoqpmW7iVCioIXHo5yYKnvin+Ne7wIGNkWxplxHDc58s='} 2024-04-16T07:33:15.423 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-16T07:33:15.431 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-04-16T07:33:15.432 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-16T07:33:15.432 INFO:teuthology.run_tasks:Running task console_log... 2024-04-16T07:33:15.509 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f13f38e9e50>, signals=[15]) 2024-04-16T07:33:15.510 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-16T07:33:15.518 INFO:teuthology.task.internal:Opening connections... 2024-04-16T07:33:15.519 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-04-16T07:33:15.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:33:15.624 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-04-16T07:33:15.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:33:15.724 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-16T07:33:15.732 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-04-16T07:33:15.751 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-04-16T07:33:15.752 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-04-16T07:33:15.810 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-04-16T07:33:15.810 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-04-16T07:33:15.810 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-04-16T07:33:15.810 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-04-16T07:33:15.810 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-04-16T07:33:15.811 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-04-16T07:33:15.811 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T07:33:15.811 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-04-16T07:33:15.811 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-04-16T07:33:15.811 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T07:33:15.812 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-04-16T07:33:15.812 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T07:33:15.812 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T07:33:15.812 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T07:33:15.813 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-04-16T07:33:15.839 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-04-16T07:33:15.856 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-04-16T07:33:15.857 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-04-16T07:33:15.914 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-04-16T07:33:15.914 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T07:33:15.915 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T07:33:15.916 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-04-16T07:33:15.939 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-16T07:33:15.949 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-16T07:33:15.956 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-16T07:33:15.957 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-04-16T07:33:15.959 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-04-16T07:33:15.975 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-16T07:33:15.983 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-16T07:33:15.983 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-04-16T07:33:16.016 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-04-16T07:33:16.040 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-16T07:33:16.093 INFO:teuthology.run_tasks:Running task kernel... 2024-04-16T07:33:16.107 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-16T07:33:16.107 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-16T07:33:16.107 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T07:33:16.108 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T07:33:16.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T07:33:16.124 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-04-16T07:33:16.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T07:33:16.125 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-04-16T07:33:16.180 INFO:teuthology.orchestra.run.smithi130.stdout:5.14.0-361.el9.x86_64 2024-04-16T07:33:16.180 INFO:teuthology.task.kernel:Running kernel on smithi130: 5.14.0-361.el9.x86_64 2024-04-16T07:33:16.180 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum install -y kernel 2024-04-16T07:33:16.182 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-361.el9.x86_64 2024-04-16T07:33:16.182 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-361.el9.x86_64 2024-04-16T07:33:16.183 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-04-16T07:33:18.291 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-04-16T07:33:18.327 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-04-16T07:33:19.836 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2024-04-16T07:33:19.968 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-04-16T07:33:20.846 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-16T07:33:23.122 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-16T07:33:27.137 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-16T07:33:28.579 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-16T07:33:29.423 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 922 kB/s | 8.0 MB 00:08 2024-04-16T07:33:29.558 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-16T07:33:31.249 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-16T07:33:34.597 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-16T07:33:36.946 INFO:teuthology.orchestra.run.smithi130.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-16T07:33:36.947 INFO:teuthology.orchestra.run.smithi130.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-16T07:33:37.045 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi130.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout:Install 4 Packages 2024-04-16T07:33:37.052 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:33:37.053 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 77 M 2024-04-16T07:33:37.053 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 123 M 2024-04-16T07:33:37.053 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:33:37.282 INFO:teuthology.orchestra.run.smithi130.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.2 MB/s | 277 kB 00:00 2024-04-16T07:33:37.791 INFO:teuthology.orchestra.run.smithi130.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 21 MB/s | 15 MB 00:00 2024-04-16T07:33:37.799 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-16T07:33:38.607 INFO:teuthology.orchestra.run.smithi130.stdout:(3/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 22 MB/s | 34 MB 00:01 2024-04-16T07:33:39.582 INFO:teuthology.orchestra.run.smithi130.stdout:(4/4): kernel-modules-core-5.14.0-438.el9.x86_6 12 MB/s | 28 MB 00:02 2024-04-16T07:33:39.586 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:33:39.587 INFO:teuthology.orchestra.run.smithi130.stdout:Total 30 MB/s | 77 MB 00:02 2024-04-16T07:33:39.587 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:33:39.955 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:33:39.955 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:33:40.241 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-16T07:33:40.779 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:33:40.780 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:33:41.426 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-16T07:33:43.075 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:33:43.701 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-16T07:33:43.801 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T07:33:46.442 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-16T07:33:46.538 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T07:33:46.670 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T07:33:48.833 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-16T07:33:48.833 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-16T07:33:48.922 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:33:48.926 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:33:48.926 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:33:48.926 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:33:48.926 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:33:48.926 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-16T07:33:48.926 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:33:48.926 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-16T07:33:48.927 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-16T07:33:48.927 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-16T07:33:48.927 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:33:48.927 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:33:48.927 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:33:48.927 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-04-16T07:33:48.927 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:33:48.928 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 77 M 2024-04-16T07:33:48.928 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 123 M 2024-04-16T07:33:48.928 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:33:49.102 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.6 MB/s | 277 kB 00:00 2024-04-16T07:33:50.102 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 13 MB/s | 15 MB 00:01 2024-04-16T07:33:50.562 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 21 MB/s | 34 MB 00:01 2024-04-16T07:33:51.019 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-modules-core-5.14.0-438.el9.x86_6 14 MB/s | 28 MB 00:01 2024-04-16T07:33:51.022 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:33:51.023 INFO:teuthology.orchestra.run.smithi164.stdout:Total 37 MB/s | 77 MB 00:02 2024-04-16T07:33:51.023 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:33:51.389 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:33:51.389 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:33:51.396 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T07:33:51.429 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:33:52.248 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:33:52.248 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:33:54.619 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:33:55.229 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-16T07:33:55.311 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T07:33:56.168 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:33:58.088 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T07:33:58.222 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T07:34:02.966 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T07:34:03.004 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:07.752 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:25.332 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:25.333 INFO:teuthology.orchestra.run.smithi130.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T07:34:25.333 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:34:25.378 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:25.718 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:25.718 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-16T07:34:25.718 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T07:34:25.719 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:34:26.144 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:34:26.477 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T07:34:26.580 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T07:34:26.580 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T07:34:26.581 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-16T07:34:26.581 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T07:34:26.581 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T07:34:26.581 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-16T07:34:26.581 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi130.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T07:34:26.581 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T07:34:26.661 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T07:34:26.661 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T07:34:26.661 DEBUG:teuthology.orchestra.run.smithi130:> sudo rpm -qi grub2-tools 2024-04-16T07:34:26.696 INFO:teuthology.orchestra.run.smithi130.stdout:Name : grub2-tools 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Epoch : 1 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Version : 2.06 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Release : 68.el9 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Architecture: x86_64 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Group : Unspecified 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Size : 8264254 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:License : GPLv3+ 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Packager : builder@centos.org 2024-04-16T07:34:26.697 INFO:teuthology.orchestra.run.smithi130.stdout:Vendor : CentOS 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:Summary : Support tools for GRUB. 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:Description : 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:hardware devices. 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:34:26.698 INFO:teuthology.orchestra.run.smithi130.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T07:34:26.699 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T07:34:26.699 INFO:teuthology.task.kernel:Updating grub on smithi130 to boot 5.14.0-438.el9.x86_64 2024-04-16T07:34:26.700 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T07:34:27.284 INFO:teuthology.orchestra.run.smithi130.stderr:Generating grub configuration file ... 2024-04-16T07:34:28.684 INFO:teuthology.orchestra.run.smithi130.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T07:34:28.714 INFO:teuthology.orchestra.run.smithi130.stderr:done 2024-04-16T07:34:28.716 DEBUG:teuthology.orchestra.run.smithi130:> mktemp 2024-04-16T07:34:28.731 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/tmp.q1PsvXIB76 2024-04-16T07:34:28.732 DEBUG:teuthology.orchestra.run.smithi130:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q1PsvXIB76 2024-04-16T07:34:28.800 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0666 /tmp/tmp.q1PsvXIB76 2024-04-16T07:34:28.940 DEBUG:teuthology.orchestra.remote:smithi130:/tmp/tmp.q1PsvXIB76 is 6KB 2024-04-16T07:34:28.989 DEBUG:teuthology.orchestra.run.smithi130:> rm -fr /tmp/tmp.q1PsvXIB76 2024-04-16T07:34:29.003 DEBUG:teuthology.orchestra.run.smithi130:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T07:34:29.068 INFO:teuthology.orchestra.run.smithi130.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T07:34:29.068 INFO:teuthology.orchestra.run.smithi130.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T07:34:29.068 INFO:teuthology.orchestra.run.smithi130.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T07:34:29.068 INFO:teuthology.orchestra.run.smithi130.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-16T07:34:29.069 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-16T07:34:29.221 DEBUG:teuthology.orchestra.run.smithi130:> sudo shutdown -r now 2024-04-16T07:34:37.439 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:37.439 INFO:teuthology.orchestra.run.smithi164.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T07:34:37.439 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:34:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:37.865 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:37.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-16T07:34:37.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T07:34:37.865 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T07:34:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T07:34:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:34:38.263 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:34:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-16T07:34:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-16T07:34:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-16T07:34:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-16T07:34:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:34:38.264 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:34:38.619 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T07:34:38.723 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T07:34:38.723 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T07:34:38.724 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-16T07:34:38.724 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T07:34:38.724 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T07:34:38.724 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-04-16T07:34:38.724 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T07:34:38.724 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T07:34:38.805 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T07:34:38.805 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T07:34:38.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-04-16T07:34:38.848 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-04-16T07:34:38.849 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-04-16T07:34:38.849 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-04-16T07:34:38.849 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 68.el9 2024-04-16T07:34:38.849 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-04-16T07:34:38.849 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T07:34:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-04-16T07:34:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8264254 2024-04-16T07:34:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-04-16T07:34:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T07:34:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T07:34:38.851 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T07:34:38.851 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T07:34:38.851 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-04-16T07:34:38.851 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-04-16T07:34:38.852 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T07:34:38.852 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-04-16T07:34:38.852 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-04-16T07:34:38.852 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:34:38.852 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T07:34:38.852 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T07:34:38.853 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T07:34:38.853 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-04-16T07:34:38.853 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:34:38.853 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T07:34:38.855 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T07:34:38.855 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-438.el9.x86_64 2024-04-16T07:34:38.855 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T07:34:39.461 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-04-16T07:34:40.841 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T07:34:40.873 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-04-16T07:34:40.875 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-04-16T07:34:40.890 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.301KR0Bwfh 2024-04-16T07:34:40.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.301KR0Bwfh 2024-04-16T07:34:40.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.301KR0Bwfh 2024-04-16T07:34:41.104 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.301KR0Bwfh is 6KB 2024-04-16T07:34:41.152 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.301KR0Bwfh 2024-04-16T07:34:41.167 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T07:34:41.234 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T07:34:41.234 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T07:34:41.234 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T07:34:41.234 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-16T07:34:41.235 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-16T07:34:41.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-04-16T07:34:59.243 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-16T07:34:59.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-16T07:34:59.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:35:11.398 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-04-16T07:35:11.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:35:11.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:35:17.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.130 2024-04-16T07:35:26.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-16T07:35:26.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:35:29.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-16T07:35:38.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:35:38.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:35:41.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-16T07:35:53.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:35:53.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:36:12.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-16T07:36:26.815 DEBUG:teuthology.orchestra.remote:timed out 2024-04-16T07:36:27.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:36:27.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:36:30.399 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-16T07:36:38.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-16T07:36:38.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:36:39.142 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-04-16T07:36:39.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-16T07:36:39.522 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-438.el9.x86_64"... 2024-04-16T07:36:39.522 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-04-16T07:36:39.538 INFO:teuthology.orchestra.run.smithi130.stdout:5.14.0-438.el9.x86_64 2024-04-16T07:36:39.538 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-16T07:36:39.538 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T07:36:39.538 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T07:36:40.539 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-16T07:36:40.539 DEBUG:teuthology.orchestra.run.smithi130:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-16T07:36:40.601 INFO:teuthology.orchestra.run.smithi130.stdout:ttyS1 2024-04-16T07:36:40.642 DEBUG:teuthology.parallel:result is None 2024-04-16T07:36:48.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:36:48.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:36:48.790 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-16T07:36:49.166 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:36:49.166 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-438.el9.x86_64"... 2024-04-16T07:36:49.167 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-04-16T07:36:49.184 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-438.el9.x86_64 2024-04-16T07:36:49.184 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-16T07:36:49.184 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T07:36:49.184 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T07:36:50.185 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-16T07:36:50.185 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-16T07:36:50.521 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-04-16T07:36:50.555 DEBUG:teuthology.parallel:result is None 2024-04-16T07:36:50.556 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-16T07:36:50.565 INFO:teuthology.task.internal:Creating test directory... 2024-04-16T07:36:50.565 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T07:36:50.568 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T07:36:50.586 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-16T07:36:50.594 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-16T07:36:50.603 INFO:teuthology.task.internal:Creating archive directory... 2024-04-16T07:36:50.603 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T07:36:50.629 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T07:36:50.965 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-16T07:36:50.973 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-16T07:36:50.974 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T07:36:50.977 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T07:36:51.017 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T07:36:51.022 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T07:36:51.031 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T07:36:51.035 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T07:36:51.036 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-16T07:36:51.045 INFO:teuthology.task.internal:Configuring sudo... 2024-04-16T07:36:51.046 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T07:36:51.077 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T07:36:51.108 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-16T07:36:51.117 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-16T07:36:51.117 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T07:36:51.148 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T07:36:51.166 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T07:36:51.249 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T07:36:51.328 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T07:36:51.371 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T07:36:51.416 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:36:51.416 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T07:36:51.484 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T07:36:51.533 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T07:36:51.591 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T07:36:51.636 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T07:36:51.683 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:36:51.683 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T07:36:51.752 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-04-16T07:36:51.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-04-16T07:36:51.801 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T07:36:51.836 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T07:36:52.227 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-16T07:36:52.236 INFO:teuthology.task.internal:Starting timer... 2024-04-16T07:36:52.236 INFO:teuthology.run_tasks:Running task pcp... 2024-04-16T07:36:52.248 INFO:teuthology.run_tasks:Running task selinux... 2024-04-16T07:36:52.258 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-16T07:36:52.258 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-04-16T07:36:52.316 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-04-16T07:36:52.318 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-04-16T07:36:52.382 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-04-16T07:36:52.384 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-16T07:36:52.385 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-04-16T07:36:52.408 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-04-16T07:36:52.408 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-04-16T07:36:52.442 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-04-16T07:36:52.443 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi130.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-04-16T07:36:52.443 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T07:36:52.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T07:36:52.481 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T07:36:52.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T07:36:52.512 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-16T07:36:52.513 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-04-16T07:36:52.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-04-16T07:36:52.595 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-16T07:36:52.604 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-16T07:36:52.626 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-16T07:36:52.628 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi130.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-16T07:41:17.516 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi130.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-04-16T07:41:17.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-16T07:41:17.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:41:17.612 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-04-16T07:41:17.685 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-16T07:41:17.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:41:17.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T07:41:17.773 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-16T07:41:17.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-16T07:41:17.850 INFO:teuthology.run_tasks:Running task clock... 2024-04-16T07:41:17.860 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-16T07:41:17.861 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T07:41:17.861 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T07:41:17.864 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T07:41:17.864 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T07:41:17.893 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T07:41:17.910 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T07:41:17.922 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T07:41:17.940 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T07:41:17.941 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-04-16T07:41:17.955 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-04-16T07:41:17.970 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-04-16T07:41:17.973 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T07:41:17.985 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-04-16T07:41:17.990 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T07:41:18.003 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T07:41:18.020 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T07:41:18.041 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-16T07:41:18.045 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T07:41:18.045 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-16T07:41:18.045 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.045 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.045 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.045 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.075 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-16T07:41:18.078 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T07:41:18.078 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-16T07:41:18.078 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.078 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.078 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.078 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T07:41:18.079 INFO:teuthology.run_tasks:Running task install... 2024-04-16T07:41:18.088 DEBUG:teuthology.task.install:project ceph 2024-04-16T07:41:18.088 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb'}} 2024-04-16T07:41:18.088 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb'} 2024-04-16T07:41:18.088 INFO:teuthology.task.install:Using flavor: crimson 2024-04-16T07:41:18.094 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-16T07:41:18.094 INFO:teuthology.task.install:extra packages: [] 2024-04-16T07:41:18.094 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb', 'tag': None, 'wait_for_package': False} 2024-04-16T07:41:18.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=30b0aa0009a925d6a46536e55c5656edcd51ecdb 2024-04-16T07:41:18.097 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb', 'tag': None, 'wait_for_package': False} 2024-04-16T07:41:18.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=30b0aa0009a925d6a46536e55c5656edcd51ecdb 2024-04-16T07:41:18.382 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/ 2024-04-16T07:41:18.383 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3109.g30b0aa00 2024-04-16T07:41:18.390 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/ 2024-04-16T07:41:18.391 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3109.g30b0aa00 2024-04-16T07:41:18.709 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T07:41:18.709 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:41:18.709 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T07:41:18.741 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-16T07:41:18.741 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/30b0aa0009a925d6a46536e55c5656edcd51ecdb/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T07:41:18.812 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T07:41:18.865 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-matanb-crimson-replicated-april-15/30b0aa0009a925d6a46536e55c5656edcd51ecdb/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T07:41:18.865 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:41:18.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T07:41:18.896 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T07:41:18.898 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-16T07:41:18.898 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/30b0aa0009a925d6a46536e55c5656edcd51ecdb/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T07:41:18.965 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-04-16T07:41:18.967 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-04-16T07:41:18.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T07:41:19.055 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T07:41:19.123 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-04-16T07:41:19.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-04-16T07:41:19.209 INFO:teuthology.orchestra.run.smithi130.stdout:74 files removed 2024-04-16T07:41:19.236 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw 2024-04-16T07:41:19.374 INFO:teuthology.orchestra.run.smithi164.stdout:74 files removed 2024-04-16T07:41:19.401 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw 2024-04-16T07:41:20.822 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 68 kB/s | 87 kB 00:01 2024-04-16T07:41:21.302 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 55 kB/s | 87 kB 00:01 2024-04-16T07:41:22.044 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 17 kB/s | 12 kB 00:00 2024-04-16T07:41:22.922 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 5.7 kB/s | 12 kB 00:02 2024-04-16T07:41:23.095 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 2.0 kB/s | 2.0 kB 00:01 2024-04-16T07:41:23.564 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-04-16T07:41:23.824 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 2.4 kB/s | 2.0 kB 00:00 2024-04-16T07:41:24.384 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-16T07:41:24.997 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-04-16T07:41:25.784 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-04-16T07:41:25.871 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-16T07:41:26.791 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-16T07:41:28.692 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-16T07:41:33.960 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-16T07:41:35.527 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-16T07:41:44.187 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-04-16T07:41:51.871 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:04 2024-04-16T07:41:53.364 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-16T07:41:54.789 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-16T07:41:58.154 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 947 kB/s | 21 MB 00:22 2024-04-16T07:41:59.915 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-16T07:42:02.661 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 10 M 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 3.4 M 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 3.7 M 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options 2024-04-16T07:42:02.674 INFO:teuthology.orchestra.run.smithi164.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 5.9 M 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 19 M 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 25 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 797 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 546 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T07:42:02.675 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 5.2 M 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 45 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 135 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 159 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 315 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 293 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 99 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T07:42:02.676 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout:Install 30 Packages 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 2 Packages 2024-04-16T07:42:02.677 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:42:02.678 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 59 M 2024-04-16T07:42:02.678 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:42:03.302 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-16T07:42:06.224 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:42:06.230 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 10 M 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 3.4 M 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 3.7 M 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 5.9 M 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 19 M 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 25 k 2024-04-16T07:42:06.231 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 797 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 546 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 5.2 M 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T07:42:06.232 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 45 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 135 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 159 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 315 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 293 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 99 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:42:06.233 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:42:06.234 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:42:06.234 INFO:teuthology.orchestra.run.smithi130.stdout:Install 30 Packages 2024-04-16T07:42:06.234 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 2 Packages 2024-04-16T07:42:06.234 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:42:06.234 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 59 M 2024-04-16T07:42:06.234 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:42:15.997 INFO:teuthology.orchestra.run.smithi164.stdout:(1/32): ceph-base-19.0.0-3109.g30b0aa00.el9.x86 466 kB/s | 5.9 MB 00:13 2024-04-16T07:42:16.098 INFO:teuthology.orchestra.run.smithi164.stdout:(2/32): ceph-selinux-19.0.0-3109.g30b0aa00.el9. 252 kB/s | 25 kB 00:00 2024-04-16T07:42:18.198 INFO:teuthology.orchestra.run.smithi164.stdout:(3/32): libcephfs2-19.0.0-3109.g30b0aa00.el9.x8 380 kB/s | 797 kB 00:02 2024-04-16T07:42:19.256 INFO:teuthology.orchestra.run.smithi164.stdout:(4/32): libradosstriper1-19.0.0-3109.g30b0aa00. 516 kB/s | 546 kB 00:01 2024-04-16T07:42:23.951 INFO:teuthology.orchestra.run.smithi130.stdout:(1/32): ceph-base-19.0.0-3109.g30b0aa00.el9.x86 350 kB/s | 5.9 MB 00:17 2024-04-16T07:42:24.076 INFO:teuthology.orchestra.run.smithi130.stdout:(2/32): ceph-selinux-19.0.0-3109.g30b0aa00.el9. 201 kB/s | 25 kB 00:00 2024-04-16T07:42:25.451 INFO:teuthology.orchestra.run.smithi130.stdout:(3/32): libcephfs2-19.0.0-3109.g30b0aa00.el9.x8 580 kB/s | 797 kB 00:01 2024-04-16T07:42:27.151 INFO:teuthology.orchestra.run.smithi130.stdout:(4/32): libradosstriper1-19.0.0-3109.g30b0aa00. 321 kB/s | 546 kB 00:01 2024-04-16T07:42:29.030 INFO:teuthology.orchestra.run.smithi164.stdout:(5/32): ceph-radosgw-19.0.0-3109.g30b0aa00.el9. 398 kB/s | 10 MB 00:26 2024-04-16T07:42:29.289 INFO:teuthology.orchestra.run.smithi164.stdout:(6/32): python3-ceph-argparse-19.0.0-3109.g30b0 174 kB/s | 45 kB 00:00 2024-04-16T07:42:29.889 INFO:teuthology.orchestra.run.smithi164.stdout:(7/32): python3-ceph-common-19.0.0-3109.g30b0aa 225 kB/s | 135 kB 00:00 2024-04-16T07:42:30.423 INFO:teuthology.orchestra.run.smithi164.stdout:(8/32): python3-cephfs-19.0.0-3109.g30b0aa00.el 299 kB/s | 159 kB 00:00 2024-04-16T07:42:30.601 INFO:teuthology.orchestra.run.smithi130.stdout:(5/32): ceph-radosgw-19.0.0-3109.g30b0aa00.el9. 432 kB/s | 10 MB 00:23 2024-04-16T07:42:30.842 INFO:teuthology.orchestra.run.smithi130.stdout:(6/32): python3-ceph-argparse-19.0.0-3109.g30b0 187 kB/s | 45 kB 00:00 2024-04-16T07:42:31.168 INFO:teuthology.orchestra.run.smithi130.stdout:(7/32): python3-ceph-common-19.0.0-3109.g30b0aa 414 kB/s | 135 kB 00:00 2024-04-16T07:42:31.764 INFO:teuthology.orchestra.run.smithi164.stdout:(9/32): python3-rados-19.0.0-3109.g30b0aa00.el9 235 kB/s | 315 kB 00:01 2024-04-16T07:42:31.926 INFO:teuthology.orchestra.run.smithi130.stdout:(8/32): python3-cephfs-19.0.0-3109.g30b0aa00.el 210 kB/s | 159 kB 00:00 2024-04-16T07:42:32.723 INFO:teuthology.orchestra.run.smithi164.stdout:(10/32): python3-rbd-19.0.0-3109.g30b0aa00.el9. 306 kB/s | 293 kB 00:00 2024-04-16T07:42:33.182 INFO:teuthology.orchestra.run.smithi164.stdout:(11/32): python3-rgw-19.0.0-3109.g30b0aa00.el9. 217 kB/s | 99 kB 00:00 2024-04-16T07:42:33.277 INFO:teuthology.orchestra.run.smithi130.stdout:(9/32): python3-rados-19.0.0-3109.g30b0aa00.el9 233 kB/s | 315 kB 00:01 2024-04-16T07:42:33.382 INFO:teuthology.orchestra.run.smithi164.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 530 kB/s | 106 kB 00:00 2024-04-16T07:42:33.490 INFO:teuthology.orchestra.run.smithi164.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-04-16T07:42:33.549 INFO:teuthology.orchestra.run.smithi164.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 759 kB/s | 44 kB 00:00 2024-04-16T07:42:33.733 INFO:teuthology.orchestra.run.smithi164.stdout:(15/32): librgw2-19.0.0-3109.g30b0aa00.el9.x86_ 371 kB/s | 5.2 MB 00:14 2024-04-16T07:42:33.791 INFO:teuthology.orchestra.run.smithi164.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 661 kB 00:00 2024-04-16T07:42:33.841 INFO:teuthology.orchestra.run.smithi164.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 282 kB 00:00 2024-04-16T07:42:33.858 INFO:teuthology.orchestra.run.smithi164.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 621 kB/s | 41 kB 00:00 2024-04-16T07:42:33.900 INFO:teuthology.orchestra.run.smithi164.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-16T07:42:33.917 INFO:teuthology.orchestra.run.smithi164.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 594 kB/s | 35 kB 00:00 2024-04-16T07:42:33.934 INFO:teuthology.orchestra.run.smithi164.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-16T07:42:34.051 INFO:teuthology.orchestra.run.smithi164.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.3 MB/s | 309 kB 00:00 2024-04-16T07:42:34.076 INFO:teuthology.orchestra.run.smithi164.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-04-16T07:42:34.110 INFO:teuthology.orchestra.run.smithi164.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-04-16T07:42:34.143 INFO:teuthology.orchestra.run.smithi164.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-04-16T07:42:34.210 INFO:teuthology.orchestra.run.smithi164.stdout:(26/32): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-16T07:42:34.261 INFO:teuthology.orchestra.run.smithi164.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-04-16T07:42:34.286 INFO:teuthology.orchestra.run.smithi164.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-16T07:42:34.318 INFO:teuthology.orchestra.run.smithi130.stdout:(10/32): python3-rbd-19.0.0-3109.g30b0aa00.el9. 281 kB/s | 293 kB 00:01 2024-04-16T07:42:34.461 INFO:teuthology.orchestra.run.smithi164.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-16T07:42:34.643 INFO:teuthology.orchestra.run.smithi130.stdout:(11/32): python3-rgw-19.0.0-3109.g30b0aa00.el9. 306 kB/s | 99 kB 00:00 2024-04-16T07:42:34.769 INFO:teuthology.orchestra.run.smithi130.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 847 kB/s | 106 kB 00:00 2024-04-16T07:42:34.819 INFO:teuthology.orchestra.run.smithi130.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-16T07:42:34.844 INFO:teuthology.orchestra.run.smithi130.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-04-16T07:42:34.920 INFO:teuthology.orchestra.run.smithi130.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-04-16T07:42:34.970 INFO:teuthology.orchestra.run.smithi130.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-16T07:42:35.004 INFO:teuthology.orchestra.run.smithi130.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 41 kB 00:00 2024-04-16T07:42:35.054 INFO:teuthology.orchestra.run.smithi130.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-04-16T07:42:35.087 INFO:teuthology.orchestra.run.smithi130.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 1.0 MB/s | 35 kB 00:00 2024-04-16T07:42:35.112 INFO:teuthology.orchestra.run.smithi130.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.8 MB/s | 46 kB 00:00 2024-04-16T07:42:35.313 INFO:teuthology.orchestra.run.smithi130.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.5 MB/s | 309 kB 00:00 2024-04-16T07:42:35.547 INFO:teuthology.orchestra.run.smithi130.stdout:(22/32): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-16T07:42:35.605 INFO:teuthology.orchestra.run.smithi130.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-04-16T07:42:35.656 INFO:teuthology.orchestra.run.smithi130.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-04-16T07:42:35.697 INFO:teuthology.orchestra.run.smithi130.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-16T07:42:35.798 INFO:teuthology.orchestra.run.smithi130.stdout:(26/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-04-16T07:42:35.840 INFO:teuthology.orchestra.run.smithi130.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-16T07:42:35.923 INFO:teuthology.orchestra.run.smithi130.stdout:(28/32): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-16T07:42:42.228 INFO:teuthology.orchestra.run.smithi164.stdout:(30/32): librados2-19.0.0-3109.g30b0aa00.el9.x8 438 kB/s | 3.4 MB 00:07 2024-04-16T07:42:43.298 INFO:teuthology.orchestra.run.smithi130.stdout:(29/32): librados2-19.0.0-3109.g30b0aa00.el9.x8 472 kB/s | 3.4 MB 00:07 2024-04-16T07:42:46.052 INFO:teuthology.orchestra.run.smithi164.stdout:(31/32): librbd1-19.0.0-3109.g30b0aa00.el9.x86_ 325 kB/s | 3.7 MB 00:11 2024-04-16T07:42:47.889 INFO:teuthology.orchestra.run.smithi130.stdout:(30/32): librgw2-19.0.0-3109.g30b0aa00.el9.x86_ 259 kB/s | 5.2 MB 00:20 2024-04-16T07:42:49.889 INFO:teuthology.orchestra.run.smithi130.stdout:(31/32): librbd1-19.0.0-3109.g30b0aa00.el9.x86_ 571 kB/s | 3.7 MB 00:06 2024-04-16T07:43:00.459 INFO:teuthology.orchestra.run.smithi164.stdout:(32/32): ceph-common-19.0.0-3109.g30b0aa00.el9. 346 kB/s | 19 MB 00:57 2024-04-16T07:43:00.462 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:43:00.463 INFO:teuthology.orchestra.run.smithi164.stdout:Total 1.0 MB/s | 59 MB 00:57 2024-04-16T07:43:00.463 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:43:00.502 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:43:00.503 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:43:00.994 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:43:00.994 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:43:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:43:02.962 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-16T07:43:03.010 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/34 2024-04-16T07:43:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/34 2024-04-16T07:43:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 3/34 2024-04-16T07:43:03.220 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/34 2024-04-16T07:43:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/34 2024-04-16T07:43:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-04-16T07:43:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-04-16T07:43:03.383 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-04-16T07:43:03.406 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-16T07:43:03.422 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-16T07:43:03.567 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-16T07:43:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-16T07:43:03.963 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-04-16T07:43:03.985 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/34 2024-04-16T07:43:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/34 2024-04-16T07:43:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3109.g30b0aa00.el9. 12/34 2024-04-16T07:43:04.166 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 13/34 2024-04-16T07:43:04.199 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 14/34 2024-04-16T07:43:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-04-16T07:43:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_6 16/34 2024-04-16T07:43:04.392 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_6 16/34 2024-04-16T07:43:04.436 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-04-16T07:43:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-04-16T07:43:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-04-16T07:43:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-04-16T07:43:04.652 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-04-16T07:43:04.681 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-04-16T07:43:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-04-16T07:43:04.755 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-16T07:43:05.319 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-16T07:43:05.351 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 26/34 2024-04-16T07:43:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 26/34 2024-04-16T07:43:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 27/34 2024-04-16T07:43:05.495 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3109.g30b0aa00.el9.x8 28/34 2024-04-16T07:43:06.745 INFO:teuthology.orchestra.run.smithi130.stdout:(32/32): ceph-common-19.0.0-3109.g30b0aa00.el9. 331 kB/s | 19 MB 01:00 2024-04-16T07:43:06.750 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:43:06.750 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.0 MB/s | 59 MB 01:00 2024-04-16T07:43:06.750 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:43:06.790 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:43:06.791 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:43:07.268 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:43:07.269 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:43:07.909 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 29/34 2024-04-16T07:43:07.950 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 29/34 2024-04-16T07:43:08.632 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 29/34 2024-04-16T07:43:08.678 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 30/34 2024-04-16T07:43:08.728 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:43:08.776 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 30/34 2024-04-16T07:43:08.777 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T07:43:08.777 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T07:43:08.777 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:43:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 31/34 2024-04-16T07:43:09.207 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-16T07:43:09.247 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/34 2024-04-16T07:43:09.322 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/34 2024-04-16T07:43:09.410 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 3/34 2024-04-16T07:43:09.441 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/34 2024-04-16T07:43:09.486 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/34 2024-04-16T07:43:09.532 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-04-16T07:43:09.561 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-04-16T07:43:09.587 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-04-16T07:43:09.610 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-16T07:43:09.625 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-16T07:43:09.811 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-16T07:43:09.842 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-16T07:43:10.192 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-04-16T07:43:10.222 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/34 2024-04-16T07:43:10.271 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/34 2024-04-16T07:43:10.315 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:19.0.0-3109.g30b0aa00.el9. 12/34 2024-04-16T07:43:10.369 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 13/34 2024-04-16T07:43:10.394 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 14/34 2024-04-16T07:43:10.472 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-04-16T07:43:10.500 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_6 16/34 2024-04-16T07:43:10.587 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_6 16/34 2024-04-16T07:43:10.624 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-04-16T07:43:10.733 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-04-16T07:43:10.767 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-04-16T07:43:10.807 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-04-16T07:43:10.839 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-04-16T07:43:10.868 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-04-16T07:43:10.900 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-04-16T07:43:10.944 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-16T07:43:11.505 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-16T07:43:11.540 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 26/34 2024-04-16T07:43:11.594 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 26/34 2024-04-16T07:43:11.663 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 27/34 2024-04-16T07:43:11.692 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:19.0.0-3109.g30b0aa00.el9.x8 28/34 2024-04-16T07:43:14.377 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 29/34 2024-04-16T07:43:14.422 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 29/34 2024-04-16T07:43:15.093 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 29/34 2024-04-16T07:43:15.133 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 30/34 2024-04-16T07:43:15.229 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 30/34 2024-04-16T07:43:15.229 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T07:43:15.229 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T07:43:15.229 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:43:15.268 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 31/34 2024-04-16T07:43:18.553 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 31/34 2024-04-16T07:43:18.553 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-04-16T07:43:18.553 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-04-16T07:43:18.553 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-04-16T07:43:18.553 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-04-16T07:43:18.553 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-04-16T07:43:18.554 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-04-16T07:43:18.554 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-04-16T07:43:18.554 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:43:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/34 2024-04-16T07:43:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/34 2024-04-16T07:43:19.674 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:43:19.674 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T07:43:19.674 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T07:43:19.674 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T07:43:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:43:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-16T07:43:19.759 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-16T07:43:19.760 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-16T07:43:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-16T07:43:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/34 2024-04-16T07:43:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/34 2024-04-16T07:43:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 3/34 2024-04-16T07:43:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 5/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_6 6/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 7/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3109.g30b0aa00.el9. 8/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3109.g30b0aa00.el9.x8 9/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 10/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 12/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 13/34 2024-04-16T07:43:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-16T07:43:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-16T07:43:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-16T07:43:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-16T07:43:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-16T07:43:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 31/34 2024-04-16T07:43:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-16T07:43:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 33/34 2024-04-16T07:43:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-16T07:43:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:43:21.867 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:43:21.870 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:43:22.128 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-test 2024-04-16T07:43:22.708 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:23 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:43:22.869 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:43:22.870 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:43:22.870 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:43:22.870 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:43:22.870 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:43:22.870 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 84 M 2024-04-16T07:43:22.870 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:43:22.870 INFO:teuthology.orchestra.run.smithi164.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 173 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:43:22.871 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:43:22.872 INFO:teuthology.orchestra.run.smithi164.stdout:Install 11 Packages 2024-04-16T07:43:22.872 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:43:22.872 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 87 M 2024-04-16T07:43:22.872 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 814 M 2024-04-16T07:43:22.872 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:43:23.746 INFO:teuthology.orchestra.run.smithi164.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-04-16T07:43:23.813 INFO:teuthology.orchestra.run.smithi164.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-16T07:43:23.846 INFO:teuthology.orchestra.run.smithi164.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 8.8 MB/s | 300 kB 00:00 2024-04-16T07:43:23.880 INFO:teuthology.orchestra.run.smithi164.stdout:(4/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-16T07:43:23.914 INFO:teuthology.orchestra.run.smithi164.stdout:(5/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.6 MB/s | 90 kB 00:00 2024-04-16T07:43:24.089 INFO:teuthology.orchestra.run.smithi164.stdout:(6/11): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-04-16T07:43:24.231 INFO:teuthology.orchestra.run.smithi164.stdout:(7/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 3.3 MB/s | 469 kB 00:00 2024-04-16T07:43:24.323 INFO:teuthology.orchestra.run.smithi164.stdout:(8/11): libubsan-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-04-16T07:43:24.506 INFO:teuthology.orchestra.run.smithi164.stdout:(9/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 665 kB/s | 122 kB 00:00 2024-04-16T07:43:24.731 INFO:teuthology.orchestra.run.smithi164.stdout:(10/11): libcephsqlite-19.0.0-3109.g30b0aa00.el 150 kB/s | 173 kB 00:01 2024-04-16T07:43:24.854 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 31/34 2024-04-16T07:43:24.854 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-04-16T07:43:24.855 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-04-16T07:43:24.855 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-04-16T07:43:24.855 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-04-16T07:43:24.855 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-04-16T07:43:24.855 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-04-16T07:43:24.855 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-04-16T07:43:24.855 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:43:25.974 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/34 2024-04-16T07:43:25.999 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/34 2024-04-16T07:43:25.999 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:43:25.999 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T07:43:25.999 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T07:43:25.999 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T07:43:26.000 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:43:26.001 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-16T07:43:26.085 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-16T07:43:26.085 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-16T07:43:27.312 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 3/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 5/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_6 6/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 7/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3109.g30b0aa00.el9. 8/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:19.0.0-3109.g30b0aa00.el9.x8 9/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 10/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/34 2024-04-16T07:43:27.313 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 12/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 13/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-16T07:43:27.314 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-16T07:43:27.315 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-16T07:43:27.315 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-16T07:43:27.315 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-16T07:43:27.315 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-16T07:43:27.316 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-16T07:43:27.316 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-16T07:43:27.316 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 31/34 2024-04-16T07:43:27.316 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-16T07:43:27.316 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 33/34 2024-04-16T07:43:28.454 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T07:43:28.455 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.456 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:43:28.457 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:43:28.678 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-test 2024-04-16T07:43:29.253 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:26 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:43:29.407 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 84 M 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 173 k 2024-04-16T07:43:29.408 INFO:teuthology.orchestra.run.smithi130.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout:Install 11 Packages 2024-04-16T07:43:29.409 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:43:29.410 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 87 M 2024-04-16T07:43:29.410 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 814 M 2024-04-16T07:43:29.410 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:43:30.523 INFO:teuthology.orchestra.run.smithi130.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 720 kB/s | 217 kB 00:00 2024-04-16T07:43:30.740 INFO:teuthology.orchestra.run.smithi130.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-04-16T07:43:30.790 INFO:teuthology.orchestra.run.smithi130.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-16T07:43:30.824 INFO:teuthology.orchestra.run.smithi130.stdout:(4/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-16T07:43:30.857 INFO:teuthology.orchestra.run.smithi130.stdout:(5/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.6 MB/s | 90 kB 00:00 2024-04-16T07:43:31.149 INFO:teuthology.orchestra.run.smithi130.stdout:(6/11): jq-1.6-16.el9.x86_64.rpm 642 kB/s | 187 kB 00:00 2024-04-16T07:43:31.200 INFO:teuthology.orchestra.run.smithi130.stdout:(7/11): libcephsqlite-19.0.0-3109.g30b0aa00.el9 177 kB/s | 173 kB 00:00 2024-04-16T07:43:31.325 INFO:teuthology.orchestra.run.smithi130.stdout:(8/11): libubsan-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-04-16T07:43:31.375 INFO:teuthology.orchestra.run.smithi130.stdout:(9/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 2.0 MB/s | 469 kB 00:00 2024-04-16T07:43:31.942 INFO:teuthology.orchestra.run.smithi130.stdout:(10/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 197 kB/s | 122 kB 00:00 2024-04-16T07:45:49.988 INFO:teuthology.orchestra.run.smithi164.stdout:(11/11): ceph-test-19.0.0-3109.g30b0aa00.el9.x8 590 kB/s | 84 MB 02:26 2024-04-16T07:45:49.990 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:45:49.990 INFO:teuthology.orchestra.run.smithi164.stdout:Total 606 kB/s | 87 MB 02:27 2024-04-16T07:45:50.023 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:45:50.045 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:45:50.046 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:45:50.674 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:45:50.675 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:45:51.705 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:45:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-04-16T07:45:51.804 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-04-16T07:45:51.830 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-16T07:45:51.907 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-16T07:45:51.938 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-04-16T07:45:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-04-16T07:45:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-04-16T07:45:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-04-16T07:45:52.091 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-04-16T07:45:52.133 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-04-16T07:45:52.160 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 10/11 2024-04-16T07:46:03.455 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 10/11 2024-04-16T07:46:03.527 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/11 2024-04-16T07:46:09.558 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/11 2024-04-16T07:46:09.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/11 2024-04-16T07:46:09.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/11 2024-04-16T07:46:09.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-04-16T07:46:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-04-16T07:46:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-04-16T07:46:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-04-16T07:46:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-04-16T07:46:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-04-16T07:46:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-04-16T07:46:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-04-16T07:46:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-04-16T07:46:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: jq-1.6-16.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: libubsan-11.4.1-3.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:46:09.972 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:46:10.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph 2024-04-16T07:46:10.857 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:04:11 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:46:11.007 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout:========================================================================================= 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout:========================================================================================= 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 6.5 k 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 2.2 M 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 1.6 M 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 249 k 2024-04-16T07:46:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 5.0 M 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 71 M 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T07:46:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T07:46:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout:========================================================================================= 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout:Install 36 Packages 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:46:11.013 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 84 M 2024-04-16T07:46:11.014 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 922 M 2024-04-16T07:46:11.014 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:46:12.129 INFO:teuthology.orchestra.run.smithi130.stdout:(11/11): ceph-test-19.0.0-3109.g30b0aa00.el9.x8 533 kB/s | 84 MB 02:41 2024-04-16T07:46:12.132 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:46:12.132 INFO:teuthology.orchestra.run.smithi130.stdout:Total 548 kB/s | 87 MB 02:42 2024-04-16T07:46:12.166 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:46:12.189 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:46:12.189 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:46:12.287 INFO:teuthology.orchestra.run.smithi164.stdout:(1/36): ceph-19.0.0-3109.g30b0aa00.el9.x86_64.r 14 kB/s | 6.5 kB 00:00 2024-04-16T07:46:12.796 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:46:12.796 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:46:13.453 INFO:teuthology.orchestra.run.smithi164.stdout:(2/36): ceph-mgr-19.0.0-3109.g30b0aa00.el9.x86_ 970 kB/s | 1.6 MB 00:01 2024-04-16T07:46:13.803 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:46:13.868 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-04-16T07:46:13.905 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-04-16T07:46:13.931 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-16T07:46:14.017 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-16T07:46:14.037 INFO:teuthology.orchestra.run.smithi164.stdout:(3/36): ceph-mds-19.0.0-3109.g30b0aa00.el9.x86_ 1.0 MB/s | 2.2 MB 00:02 2024-04-16T07:46:14.056 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-04-16T07:46:14.099 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-04-16T07:46:14.144 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-04-16T07:46:14.179 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-04-16T07:46:14.219 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-04-16T07:46:14.237 INFO:teuthology.orchestra.run.smithi164.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3109.g30b0 1.2 MB/s | 249 kB 00:00 2024-04-16T07:46:14.261 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-04-16T07:46:14.306 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 10/11 2024-04-16T07:46:14.363 INFO:teuthology.orchestra.run.smithi164.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-04-16T07:46:14.404 INFO:teuthology.orchestra.run.smithi164.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-04-16T07:46:14.455 INFO:teuthology.orchestra.run.smithi164.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 626 kB/s | 32 kB 00:00 2024-04-16T07:46:14.488 INFO:teuthology.orchestra.run.smithi164.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-04-16T07:46:14.522 INFO:teuthology.orchestra.run.smithi164.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-04-16T07:46:14.564 INFO:teuthology.orchestra.run.smithi164.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-04-16T07:46:14.631 INFO:teuthology.orchestra.run.smithi164.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-04-16T07:46:14.665 INFO:teuthology.orchestra.run.smithi164.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-04-16T07:46:14.698 INFO:teuthology.orchestra.run.smithi164.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.7 MB/s | 124 kB 00:00 2024-04-16T07:46:14.740 INFO:teuthology.orchestra.run.smithi164.stdout:(14/36): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-04-16T07:46:15.082 INFO:teuthology.orchestra.run.smithi164.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 366 kB/s | 125 kB 00:00 2024-04-16T07:46:15.308 INFO:teuthology.orchestra.run.smithi164.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-04-16T07:46:16.000 INFO:teuthology.orchestra.run.smithi164.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 218 kB/s | 151 kB 00:00 2024-04-16T07:46:16.158 INFO:teuthology.orchestra.run.smithi164.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 275 kB/s | 43 kB 00:00 2024-04-16T07:46:16.458 INFO:teuthology.orchestra.run.smithi164.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 573 kB/s | 172 kB 00:00 2024-04-16T07:46:16.784 INFO:teuthology.orchestra.run.smithi164.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.1 MB/s | 358 kB 00:00 2024-04-16T07:46:16.951 INFO:teuthology.orchestra.run.smithi164.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 64 kB/s | 11 kB 00:00 2024-04-16T07:46:17.109 INFO:teuthology.orchestra.run.smithi164.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 112 kB/s | 18 kB 00:00 2024-04-16T07:46:17.268 INFO:teuthology.orchestra.run.smithi164.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 146 kB/s | 23 kB 00:00 2024-04-16T07:46:17.426 INFO:teuthology.orchestra.run.smithi164.stdout:(24/36): python3-jaraco-functools-3.5.0-2.el9.n 123 kB/s | 19 kB 00:00 2024-04-16T07:46:17.585 INFO:teuthology.orchestra.run.smithi164.stdout:(25/36): python3-jaraco-text-3.2.0-6.el9.noarch 124 kB/s | 20 kB 00:00 2024-04-16T07:46:17.744 INFO:teuthology.orchestra.run.smithi164.stdout:(26/36): python3-logutils-0.3.5-21.el9.noarch.r 292 kB/s | 46 kB 00:00 2024-04-16T07:46:17.902 INFO:teuthology.orchestra.run.smithi164.stdout:(27/36): python3-more-itertools-8.12.0-2.el9.no 498 kB/s | 79 kB 00:00 2024-04-16T07:46:18.069 INFO:teuthology.orchestra.run.smithi164.stdout:(28/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-16T07:46:18.228 INFO:teuthology.orchestra.run.smithi164.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 104 kB/s | 16 kB 00:00 2024-04-16T07:46:18.395 INFO:teuthology.orchestra.run.smithi164.stdout:(30/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 540 kB/s | 90 kB 00:00 2024-04-16T07:46:18.553 INFO:teuthology.orchestra.run.smithi164.stdout:(31/36): python3-tempora-5.0.0-2.el9.noarch.rpm 226 kB/s | 36 kB 00:00 2024-04-16T07:46:18.712 INFO:teuthology.orchestra.run.smithi164.stdout:(32/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-04-16T07:46:18.913 INFO:teuthology.orchestra.run.smithi164.stdout:(33/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.1 MB/s | 427 kB 00:00 2024-04-16T07:46:19.071 INFO:teuthology.orchestra.run.smithi164.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 126 kB/s | 20 kB 00:00 2024-04-16T07:46:25.187 INFO:teuthology.orchestra.run.smithi164.stdout:(35/36): ceph-mon-19.0.0-3109.g30b0aa00.el9.x86 399 kB/s | 5.0 MB 00:12 2024-04-16T07:46:25.551 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 10/11 2024-04-16T07:46:25.628 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/11 2024-04-16T07:46:31.665 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 11/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-04-16T07:46:31.666 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-04-16T07:46:32.069 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: jq-1.6-16.el9.x86_64 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: libubsan-11.4.1-3.el9.x86_64 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T07:46:32.070 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T07:46:32.071 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T07:46:32.071 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T07:46:32.071 INFO:teuthology.orchestra.run.smithi130.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-04-16T07:46:32.071 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:46:32.071 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:46:32.352 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph 2024-04-16T07:46:32.939 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:04:29 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:46:33.095 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:46:33.097 INFO:teuthology.orchestra.run.smithi130.stdout:========================================================================================= 2024-04-16T07:46:33.097 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:46:33.097 INFO:teuthology.orchestra.run.smithi130.stdout:========================================================================================= 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 6.5 k 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 2.2 M 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 1.6 M 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 249 k 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 5.0 M 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 71 M 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T07:46:33.098 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T07:46:33.099 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:46:33.100 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:46:33.101 INFO:teuthology.orchestra.run.smithi130.stdout:========================================================================================= 2024-04-16T07:46:33.101 INFO:teuthology.orchestra.run.smithi130.stdout:Install 36 Packages 2024-04-16T07:46:33.101 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:46:33.101 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 84 M 2024-04-16T07:46:33.101 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 922 M 2024-04-16T07:46:33.101 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:46:34.045 INFO:teuthology.orchestra.run.smithi130.stdout:(1/36): ceph-19.0.0-3109.g30b0aa00.el9.x86_64.r 14 kB/s | 6.5 kB 00:00 2024-04-16T07:46:36.436 INFO:teuthology.orchestra.run.smithi130.stdout:(2/36): ceph-mds-19.0.0-3109.g30b0aa00.el9.x86_ 784 kB/s | 2.2 MB 00:02 2024-04-16T07:46:37.536 INFO:teuthology.orchestra.run.smithi130.stdout:(3/36): ceph-mgr-19.0.0-3109.g30b0aa00.el9.x86_ 403 kB/s | 1.6 MB 00:03 2024-04-16T07:46:38.345 INFO:teuthology.orchestra.run.smithi130.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3109.g30b0 308 kB/s | 249 kB 00:00 2024-04-16T07:46:38.462 INFO:teuthology.orchestra.run.smithi130.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-04-16T07:46:38.504 INFO:teuthology.orchestra.run.smithi130.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-04-16T07:46:38.529 INFO:teuthology.orchestra.run.smithi130.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-04-16T07:46:38.554 INFO:teuthology.orchestra.run.smithi130.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.9 MB/s | 47 kB 00:00 2024-04-16T07:46:38.588 INFO:teuthology.orchestra.run.smithi130.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-04-16T07:46:38.629 INFO:teuthology.orchestra.run.smithi130.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-04-16T07:46:38.688 INFO:teuthology.orchestra.run.smithi130.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 20 MB/s | 1.1 MB 00:00 2024-04-16T07:46:38.722 INFO:teuthology.orchestra.run.smithi130.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-04-16T07:46:38.755 INFO:teuthology.orchestra.run.smithi130.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.7 MB/s | 124 kB 00:00 2024-04-16T07:46:38.789 INFO:teuthology.orchestra.run.smithi130.stdout:(14/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-04-16T07:46:38.931 INFO:teuthology.orchestra.run.smithi130.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 881 kB/s | 125 kB 00:00 2024-04-16T07:46:38.973 INFO:teuthology.orchestra.run.smithi130.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 537 kB/s | 22 kB 00:00 2024-04-16T07:46:39.181 INFO:teuthology.orchestra.run.smithi130.stdout:(17/36): ceph-mon-19.0.0-3109.g30b0aa00.el9.x86 1.0 MB/s | 5.0 MB 00:05 2024-04-16T07:46:39.207 INFO:teuthology.orchestra.run.smithi130.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 647 kB/s | 151 kB 00:00 2024-04-16T07:46:39.299 INFO:teuthology.orchestra.run.smithi130.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-16T07:46:39.340 INFO:teuthology.orchestra.run.smithi130.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 274 kB/s | 43 kB 00:00 2024-04-16T07:46:39.399 INFO:teuthology.orchestra.run.smithi130.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.5 MB/s | 358 kB 00:00 2024-04-16T07:46:39.416 INFO:teuthology.orchestra.run.smithi130.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-04-16T07:46:39.450 INFO:teuthology.orchestra.run.smithi130.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-04-16T07:46:39.475 INFO:teuthology.orchestra.run.smithi130.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 394 kB/s | 23 kB 00:00 2024-04-16T07:46:39.501 INFO:teuthology.orchestra.run.smithi130.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-04-16T07:46:39.534 INFO:teuthology.orchestra.run.smithi130.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 336 kB/s | 20 kB 00:00 2024-04-16T07:46:39.551 INFO:teuthology.orchestra.run.smithi130.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-04-16T07:46:39.618 INFO:teuthology.orchestra.run.smithi130.stdout:(28/36): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-16T07:46:39.643 INFO:teuthology.orchestra.run.smithi130.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 724 kB/s | 79 kB 00:00 2024-04-16T07:46:39.668 INFO:teuthology.orchestra.run.smithi130.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-04-16T07:46:39.727 INFO:teuthology.orchestra.run.smithi130.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.1 MB/s | 90 kB 00:00 2024-04-16T07:46:39.744 INFO:teuthology.orchestra.run.smithi130.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 476 kB/s | 36 kB 00:00 2024-04-16T07:46:39.844 INFO:teuthology.orchestra.run.smithi130.stdout:(33/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-04-16T07:46:39.869 INFO:teuthology.orchestra.run.smithi130.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-04-16T07:46:39.894 INFO:teuthology.orchestra.run.smithi130.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-04-16T07:47:27.089 INFO:teuthology.orchestra.run.smithi164.stdout:(36/36): ceph-osd-19.0.0-3109.g30b0aa00.el9.x86 986 kB/s | 71 MB 01:13 2024-04-16T07:47:27.092 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:47:27.092 INFO:teuthology.orchestra.run.smithi164.stdout:Total 1.1 MB/s | 84 MB 01:16 2024-04-16T07:47:27.112 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:47:27.154 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:47:27.154 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:47:27.889 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:47:27.889 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:47:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:47:29.574 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-16T07:47:29.614 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-16T07:47:29.642 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-16T07:47:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-16T07:47:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-16T07:47:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-16T07:47:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-16T07:47:29.961 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-16T07:47:30.010 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-16T07:47:30.083 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-16T07:47:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-04-16T07:47:30.182 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-04-16T07:47:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-04-16T07:47:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-04-16T07:47:30.477 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-04-16T07:47:30.528 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-04-16T07:47:30.562 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-04-16T07:47:30.602 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-04-16T07:47:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-04-16T07:47:30.708 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-04-16T07:47:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-16T07:47:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-04-16T07:47:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-04-16T07:47:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-04-16T07:47:30.885 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-04-16T07:47:30.975 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-04-16T07:47:31.016 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-04-16T07:47:31.096 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-04-16T07:47:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-04-16T07:47:31.360 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-16T07:47:31.544 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3109.g30b0aa00.el9. 31/36 2024-04-16T07:47:31.571 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/36 2024-04-16T07:47:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/36 2024-04-16T07:47:31.597 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:31.597 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T07:47:31.597 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T07:47:31.597 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T07:47:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:36.188 INFO:teuthology.orchestra.run.smithi130.stdout:(36/36): ceph-osd-19.0.0-3109.g30b0aa00.el9.x86 1.2 MB/s | 71 MB 00:59 2024-04-16T07:47:36.191 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:47:36.191 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.3 MB/s | 84 MB 01:03 2024-04-16T07:47:36.211 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:47:36.252 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:47:36.252 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:47:36.999 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:47:37.000 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:47:38.637 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:47:38.698 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-16T07:47:38.740 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-16T07:47:38.775 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-16T07:47:38.846 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-16T07:47:38.891 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-16T07:47:38.923 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-16T07:47:39.015 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-16T07:47:39.094 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-16T07:47:39.134 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-16T07:47:39.206 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-16T07:47:39.268 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-04-16T07:47:39.307 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-04-16T07:47:39.355 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-04-16T07:47:39.410 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-04-16T07:47:39.603 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-04-16T07:47:39.652 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-04-16T07:47:39.688 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-04-16T07:47:39.726 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-04-16T07:47:39.755 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-04-16T07:47:39.831 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-04-16T07:47:39.866 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-16T07:47:39.899 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-04-16T07:47:39.940 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-04-16T07:47:39.974 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-04-16T07:47:40.009 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-04-16T07:47:40.098 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-04-16T07:47:40.131 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-04-16T07:47:40.194 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-04-16T07:47:40.331 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-04-16T07:47:40.457 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-16T07:47:40.649 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3109.g30b0aa00.el9. 31/36 2024-04-16T07:47:40.677 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/36 2024-04-16T07:47:40.703 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 32/36 2024-04-16T07:47:40.704 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:40.704 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T07:47:40.704 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T07:47:40.704 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T07:47:40.704 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:47:42.362 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 33/36 2024-04-16T07:47:42.389 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 33/36 2024-04-16T07:47:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T07:47:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T07:47:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T07:47:42.390 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:42.930 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 34/36 2024-04-16T07:47:42.953 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 34/36 2024-04-16T07:47:42.953 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:42.953 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T07:47:42.953 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T07:47:42.954 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T07:47:42.954 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 35/36 2024-04-16T07:47:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 35/36 2024-04-16T07:47:43.212 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:43.212 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T07:47:43.212 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T07:47:43.212 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T07:47:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 36/36 2024-04-16T07:47:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 36/36 2024-04-16T07:47:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/36 2024-04-16T07:47:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/36 2024-04-16T07:47:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 3/36 2024-04-16T07:47:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/36 2024-04-16T07:47:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 5/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3109.g30b0aa00.el9. 6/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-04-16T07:47:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-16T07:47:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-16T07:47:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-16T07:47:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-16T07:47:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:50.828 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:47:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:47:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:47:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T07:47:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T07:47:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T07:47:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T07:47:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T07:47:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T07:47:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T07:47:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:50.831 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:47:51.122 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-base 2024-04-16T07:47:51.427 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 33/36 2024-04-16T07:47:51.453 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 33/36 2024-04-16T07:47:51.454 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:51.454 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T07:47:51.454 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T07:47:51.454 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T07:47:51.454 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:47:51.699 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:05:52 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:47:51.797 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:47:51.849 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:47:51.850 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:47:51.850 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:47:51.911 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install cephadm 2024-04-16T07:47:51.978 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 34/36 2024-04-16T07:47:52.001 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 34/36 2024-04-16T07:47:52.001 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:52.001 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T07:47:52.001 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T07:47:52.001 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T07:47:52.001 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:47:52.237 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 35/36 2024-04-16T07:47:52.260 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 35/36 2024-04-16T07:47:52.260 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:52.260 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T07:47:52.260 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T07:47:52.260 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T07:47:52.260 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:47:52.326 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 36/36 2024-04-16T07:47:52.483 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:05:53 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:47:52.634 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:47:52.635 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 762 k 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 762 k 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 769 k 2024-04-16T07:47:52.636 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:47:52.926 INFO:teuthology.orchestra.run.smithi164.stdout:cephadm-19.0.0-3109.g30b0aa00.el9.noarch.rpm 2.6 MB/s | 762 kB 00:00 2024-04-16T07:47:52.926 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:47:52.926 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.6 MB/s | 762 kB 00:00 2024-04-16T07:47:52.927 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:47:52.932 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:47:52.932 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:47:52.938 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:47:52.938 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:47:53.012 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:47:53.402 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:47:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:47:53.860 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:47:54.143 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:47:54.143 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:54.143 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:47:54.143 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:47:54.143 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:54.144 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:47:54.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-immutable-object-cache 2024-04-16T07:47:54.942 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:05:55 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:47:55.099 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:47:55.099 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 157 k 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 157 k 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 640 k 2024-04-16T07:47:55.100 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:47:55.324 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-immutable-object-cache-19.0.0-3109.g30b0aa 704 kB/s | 157 kB 00:00 2024-04-16T07:47:55.325 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:47:55.325 INFO:teuthology.orchestra.run.smithi164.stdout:Total 699 kB/s | 157 kB 00:00 2024-04-16T07:47:55.325 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:47:55.334 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:47:55.334 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:47:55.387 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:47:55.387 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:47:55.530 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:47:55.608 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00. 1/1 2024-04-16T07:47:55.634 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00. 1/1 2024-04-16T07:47:55.634 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:47:55.634 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T07:47:55.634 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:56.618 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00. 1/1 2024-04-16T07:47:56.618 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:56.619 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:47:56.619 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:47:56.619 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:56.619 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:47:56.848 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr 2024-04-16T07:47:57.436 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:05:58 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:47:57.536 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:47:57.588 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:47:57.589 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:47:57.590 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:47:57.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-dashboard 2024-04-16T07:47:58.227 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:05:59 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:47:58.378 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:47:58.379 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-04-16T07:47:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:47:58.379 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 3.5 M 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 34 k 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 16 k 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T07:47:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T07:47:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T07:47:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:58.381 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:47:58.381 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-04-16T07:47:58.381 INFO:teuthology.orchestra.run.smithi164.stdout:Install 11 Packages 2024-04-16T07:47:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:47:58.381 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 7.6 M 2024-04-16T07:47:58.382 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 100 M 2024-04-16T07:47:58.382 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:47:59.008 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 36/36 2024-04-16T07:47:59.008 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/36 2024-04-16T07:47:59.008 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/36 2024-04-16T07:47:59.008 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 3/36 2024-04-16T07:47:59.008 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 4/36 2024-04-16T07:47:59.008 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 5/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3109.g30b0aa00.el9. 6/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-16T07:47:59.009 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-16T07:47:59.010 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-16T07:47:59.010 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-16T07:47:59.010 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-16T07:47:59.010 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-16T07:47:59.010 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-16T07:47:59.010 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-16T07:47:59.011 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-16T07:47:59.011 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-16T07:47:59.011 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-16T07:47:59.011 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-16T07:47:59.012 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-16T07:47:59.012 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-16T07:47:59.012 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-16T07:47:59.012 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-16T07:47:59.012 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-16T07:47:59.012 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-16T07:47:59.012 INFO:teuthology.orchestra.run.smithi164.stdout:(1/11): ceph-prometheus-alerts-19.0.0-3109.g30b 88 kB/s | 16 kB 00:00 2024-04-16T07:47:59.137 INFO:teuthology.orchestra.run.smithi164.stdout:(2/11): ceph-grafana-dashboards-19.0.0-3109.g30 109 kB/s | 34 kB 00:00 2024-04-16T07:47:59.162 INFO:teuthology.orchestra.run.smithi164.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 238 kB 00:00 2024-04-16T07:47:59.329 INFO:teuthology.orchestra.run.smithi164.stdout:(4/11): ceph-mgr-dashboard-19.0.0-3109.g30b0aa0 7.0 MB/s | 3.5 MB 00:00 2024-04-16T07:47:59.471 INFO:teuthology.orchestra.run.smithi164.stdout:(5/11): protobuf-compiler-3.14.0-13.el9.x86_64. 2.5 MB/s | 863 kB 00:00 2024-04-16T07:47:59.771 INFO:teuthology.orchestra.run.smithi164.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 44 kB/s | 19 kB 00:00 2024-04-16T07:48:00.110 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T07:48:00.111 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T07:48:00.112 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T07:48:00.113 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T07:48:00.114 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T07:48:00.114 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T07:48:00.114 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T07:48:00.114 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:00.114 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:00.114 INFO:teuthology.orchestra.run.smithi164.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 422 kB/s | 144 kB 00:00 2024-04-16T07:48:00.213 INFO:teuthology.orchestra.run.smithi164.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 308 kB/s | 31 kB 00:00 2024-04-16T07:48:00.264 INFO:teuthology.orchestra.run.smithi164.stdout:(9/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 498 kB/s | 548 kB 00:01 2024-04-16T07:48:00.386 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-base 2024-04-16T07:48:00.589 INFO:teuthology.orchestra.run.smithi164.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 502 kB/s | 188 kB 00:00 2024-04-16T07:48:00.965 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:05:57 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:01.062 INFO:teuthology.orchestra.run.smithi130.stdout:Package ceph-base-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:01.115 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:01.116 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:01.116 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:01.183 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install cephadm 2024-04-16T07:48:01.760 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:05:58 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:01.911 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 762 k 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 762 k 2024-04-16T07:48:01.912 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 769 k 2024-04-16T07:48:01.913 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:02.614 INFO:teuthology.orchestra.run.smithi164.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 664 kB/s | 2.0 MB 00:03 2024-04-16T07:48:02.620 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:02.621 INFO:teuthology.orchestra.run.smithi164.stdout:Total 1.8 MB/s | 7.6 MB 00:04 2024-04-16T07:48:02.641 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:02.661 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:02.661 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:02.839 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:02.839 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:02.873 INFO:teuthology.orchestra.run.smithi130.stdout:cephadm-19.0.0-3109.g30b0aa00.el9.noarch.rpm 793 kB/s | 762 kB 00:00 2024-04-16T07:48:02.874 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:02.874 INFO:teuthology.orchestra.run.smithi130.stdout:Total 792 kB/s | 762 kB 00:00 2024-04-16T07:48:02.875 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:02.880 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:02.880 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:02.886 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:02.886 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:02.958 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:03.308 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:48:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:03.386 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:48:03.419 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-04-16T07:48:03.450 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-04-16T07:48:03.546 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-04-16T07:48:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-04-16T07:48:03.712 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-04-16T07:48:03.797 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:48:03.797 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-04-16T07:48:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-04-16T07:48:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-04-16T07:48:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3109.g30b0aa00.el9 9/11 2024-04-16T07:48:04.075 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/1 2024-04-16T07:48:04.075 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:04.075 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:04.075 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:04.075 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:04.075 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:04.268 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-immutable-object-cache 2024-04-16T07:48:04.861 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:01 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:04.951 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3109.g30b0aa00.el 10/11 2024-04-16T07:48:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noa 11/11 2024-04-16T07:48:05.013 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repo Size 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 157 k 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:05.014 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-04-16T07:48:05.015 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:05.015 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 157 k 2024-04-16T07:48:05.015 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 640 k 2024-04-16T07:48:05.015 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:05.387 INFO:teuthology.orchestra.run.smithi130.stdout:ceph-immutable-object-cache-19.0.0-3109.g30b0aa 423 kB/s | 157 kB 00:00 2024-04-16T07:48:05.387 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:05.388 INFO:teuthology.orchestra.run.smithi130.stdout:Total 421 kB/s | 157 kB 00:00 2024-04-16T07:48:05.388 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:05.397 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:05.397 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:05.450 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:05.450 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:05.591 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:05.646 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00. 1/1 2024-04-16T07:48:05.672 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00. 1/1 2024-04-16T07:48:05.672 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:48:05.672 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T07:48:05.672 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:06.396 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noa 11/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3109.g30b0aa00.el 1/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noa 2/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3109.g30b0aa00.el9 3/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-04-16T07:48:06.397 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-04-16T07:48:06.690 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00. 1/1 2024-04-16T07:48:06.690 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:06.690 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:06.690 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:06.690 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:06.690 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:06.742 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T07:48:06.743 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:06.744 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:06.908 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr 2024-04-16T07:48:07.003 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-16T07:48:07.489 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:04 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:07.584 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:08 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:07.589 INFO:teuthology.orchestra.run.smithi130.stdout:Package ceph-mgr-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:07.641 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:07.642 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:07.642 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:07.707 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-dashboard 2024-04-16T07:48:07.738 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:07.739 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-04-16T07:48:07.739 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:48:07.739 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-04-16T07:48:07.739 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 7.4 M 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T07:48:07.740 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T07:48:07.741 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T07:48:07.741 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T07:48:07.741 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:07.741 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:07.741 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-04-16T07:48:07.741 INFO:teuthology.orchestra.run.smithi164.stdout:Install 12 Packages 2024-04-16T07:48:07.741 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:07.742 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 37 M 2024-04-16T07:48:07.742 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 209 M 2024-04-16T07:48:07.742 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:08.264 INFO:teuthology.orchestra.run.smithi164.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 241 kB/s | 32 kB 00:00 2024-04-16T07:48:08.285 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:05 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:08.297 INFO:teuthology.orchestra.run.smithi164.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 537 kB/s | 17 kB 00:00 2024-04-16T07:48:08.339 INFO:teuthology.orchestra.run.smithi164.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 854 kB/s | 34 kB 00:00 2024-04-16T07:48:08.398 INFO:teuthology.orchestra.run.smithi164.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-04-16T07:48:08.439 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:08.440 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-04-16T07:48:08.440 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:48:08.440 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 3.5 M 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 34 k 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 16 k 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T07:48:08.441 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout:Install 11 Packages 2024-04-16T07:48:08.442 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:08.443 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 7.6 M 2024-04-16T07:48:08.443 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 100 M 2024-04-16T07:48:08.443 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:08.573 INFO:teuthology.orchestra.run.smithi164.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-04-16T07:48:08.640 INFO:teuthology.orchestra.run.smithi164.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-04-16T07:48:08.665 INFO:teuthology.orchestra.run.smithi164.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-04-16T07:48:08.999 INFO:teuthology.orchestra.run.smithi164.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-04-16T07:48:09.164 INFO:teuthology.orchestra.run.smithi130.stdout:(1/11): ceph-grafana-dashboards-19.0.0-3109.g30 167 kB/s | 34 kB 00:00 2024-04-16T07:48:09.189 INFO:teuthology.orchestra.run.smithi130.stdout:(2/11): ceph-prometheus-alerts-19.0.0-3109.g30b 72 kB/s | 16 kB 00:00 2024-04-16T07:48:09.199 INFO:teuthology.orchestra.run.smithi164.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 977 kB/s | 195 kB 00:00 2024-04-16T07:48:09.233 INFO:teuthology.orchestra.run.smithi164.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-16T07:48:09.356 INFO:teuthology.orchestra.run.smithi130.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 238 kB 00:00 2024-04-16T07:48:09.382 INFO:teuthology.orchestra.run.smithi130.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 4.4 MB/s | 863 kB 00:00 2024-04-16T07:48:09.450 INFO:teuthology.orchestra.run.smithi164.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 980 kB/s | 245 kB 00:00 2024-04-16T07:48:09.682 INFO:teuthology.orchestra.run.smithi130.stdout:(5/11): ceph-mgr-dashboard-19.0.0-3109.g30b0aa0 4.9 MB/s | 3.5 MB 00:00 2024-04-16T07:48:09.759 INFO:teuthology.orchestra.run.smithi164.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-3 4.5 MB/s | 7.4 MB 00:01 2024-04-16T07:48:09.761 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:09.761 INFO:teuthology.orchestra.run.smithi164.stdout:Total 18 MB/s | 37 MB 00:02 2024-04-16T07:48:09.765 INFO:teuthology.orchestra.run.smithi130.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-04-16T07:48:09.805 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:09.829 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:09.829 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:10.082 INFO:teuthology.orchestra.run.smithi130.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 456 kB/s | 144 kB 00:00 2024-04-16T07:48:10.108 INFO:teuthology.orchestra.run.smithi130.stdout:(8/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 730 kB/s | 548 kB 00:00 2024-04-16T07:48:10.182 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:10.182 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:10.208 INFO:teuthology.orchestra.run.smithi130.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 245 kB/s | 31 kB 00:00 2024-04-16T07:48:10.233 INFO:teuthology.orchestra.run.smithi130.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-04-16T07:48:10.758 INFO:teuthology.orchestra.run.smithi130.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 1.9 MB/s | 2.0 MB 00:01 2024-04-16T07:48:10.764 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:10.764 INFO:teuthology.orchestra.run.smithi130.stdout:Total 3.3 MB/s | 7.6 MB 00:02 2024-04-16T07:48:10.784 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:10.802 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:10.802 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:10.880 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:10.965 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-16T07:48:10.974 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:10.975 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:10.996 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-16T07:48:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-16T07:48:11.125 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-16T07:48:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-16T07:48:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-16T07:48:11.409 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-16T07:48:11.457 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:11.539 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-04-16T07:48:11.570 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-04-16T07:48:11.660 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-04-16T07:48:11.785 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-04-16T07:48:11.839 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-04-16T07:48:11.922 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-04-16T07:48:11.966 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-04-16T07:48:11.994 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-04-16T07:48:12.027 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3109.g30b0aa00.el9 9/11 2024-04-16T07:48:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-16T07:48:12.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-16T07:48:13.071 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3109.g30b0aa00.el 10/11 2024-04-16T07:48:13.110 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noa 11/11 2024-04-16T07:48:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-16T07:48:14.423 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-16T07:48:14.449 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0a 12/12 2024-04-16T07:48:14.497 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noa 11/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3109.g30b0aa00.el 1/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noa 2/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3109.g30b0aa00.el9 3/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-04-16T07:48:14.498 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T07:48:14.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T07:48:14.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T07:48:14.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T07:48:14.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T07:48:14.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T07:48:14.841 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:14.841 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:15.082 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-16T07:48:15.668 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:12 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:15.822 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:15.823 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================ 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================ 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 7.4 M 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T07:48:15.824 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout:============================================================================================ 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout:Install 12 Packages 2024-04-16T07:48:15.825 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:15.826 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 37 M 2024-04-16T07:48:15.826 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 209 M 2024-04-16T07:48:15.826 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:16.299 INFO:teuthology.orchestra.run.smithi130.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 193 kB/s | 32 kB 00:00 2024-04-16T07:48:16.332 INFO:teuthology.orchestra.run.smithi130.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 534 kB/s | 17 kB 00:00 2024-04-16T07:48:16.382 INFO:teuthology.orchestra.run.smithi130.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 698 kB/s | 34 kB 00:00 2024-04-16T07:48:16.441 INFO:teuthology.orchestra.run.smithi130.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.8 MB/s | 3.0 MB 00:00 2024-04-16T07:48:16.725 INFO:teuthology.orchestra.run.smithi130.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2024-04-16T07:48:16.783 INFO:teuthology.orchestra.run.smithi130.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 4.8 MB 00:00 2024-04-16T07:48:16.808 INFO:teuthology.orchestra.run.smithi130.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.6 MB/s | 217 kB 00:00 2024-04-16T07:48:16.982 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0a 12/12 2024-04-16T07:48:16.982 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0a 1/12 2024-04-16T07:48:16.982 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-16T07:48:16.982 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-16T07:48:16.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-16T07:48:17.034 INFO:teuthology.orchestra.run.smithi130.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-04-16T07:48:17.084 INFO:teuthology.orchestra.run.smithi130.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 195 kB 00:00 2024-04-16T07:48:17.234 INFO:teuthology.orchestra.run.smithi130.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T07:48:17.411 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T07:48:17.412 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T07:48:17.412 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T07:48:17.412 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T07:48:17.412 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:17.412 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:17.759 INFO:teuthology.orchestra.run.smithi130.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 364 kB/s | 245 kB 00:00 2024-04-16T07:48:17.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-rook 2024-04-16T07:48:18.453 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:19 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:18.607 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:18.608 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================== 2024-04-16T07:48:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:48:18.608 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================== 2024-04-16T07:48:18.608 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 49 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:18.609 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:18.610 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================== 2024-04-16T07:48:18.610 INFO:teuthology.orchestra.run.smithi164.stdout:Install 13 Packages 2024-04-16T07:48:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:18.610 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 2.0 M 2024-04-16T07:48:18.610 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 26 M 2024-04-16T07:48:18.610 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:18.726 INFO:teuthology.orchestra.run.smithi130.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-3 2.9 MB/s | 7.4 MB 00:02 2024-04-16T07:48:18.728 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:18.729 INFO:teuthology.orchestra.run.smithi130.stdout:Total 13 MB/s | 37 MB 00:02 2024-04-16T07:48:18.775 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:18.800 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:18.800 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:19.115 INFO:teuthology.orchestra.run.smithi164.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 158 kB/s | 24 kB 00:00 2024-04-16T07:48:19.140 INFO:teuthology.orchestra.run.smithi164.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 101 kB/s | 18 kB 00:00 2024-04-16T07:48:19.154 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:19.154 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:19.166 INFO:teuthology.orchestra.run.smithi164.stdout:(3/13): ceph-mgr-rook-19.0.0-3109.g30b0aa00.el9 240 kB/s | 49 kB 00:00 2024-04-16T07:48:19.224 INFO:teuthology.orchestra.run.smithi164.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 174 kB 00:00 2024-04-16T07:48:19.250 INFO:teuthology.orchestra.run.smithi164.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-04-16T07:48:19.300 INFO:teuthology.orchestra.run.smithi164.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 211 kB 00:00 2024-04-16T07:48:19.350 INFO:teuthology.orchestra.run.smithi164.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 341 kB/s | 43 kB 00:00 2024-04-16T07:48:19.442 INFO:teuthology.orchestra.run.smithi164.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 167 kB/s | 32 kB 00:00 2024-04-16T07:48:19.467 INFO:teuthology.orchestra.run.smithi164.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 104 kB/s | 15 kB 00:00 2024-04-16T07:48:19.568 INFO:teuthology.orchestra.run.smithi164.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2024-04-16T07:48:19.593 INFO:teuthology.orchestra.run.smithi164.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 927 kB/s | 203 kB 00:00 2024-04-16T07:48:19.710 INFO:teuthology.orchestra.run.smithi164.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 633 kB/s | 90 kB 00:00 2024-04-16T07:48:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.2 MB/s | 1.0 MB 00:00 2024-04-16T07:48:19.767 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:19.767 INFO:teuthology.orchestra.run.smithi164.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2024-04-16T07:48:19.767 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:19.777 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:19.777 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:19.854 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:19.867 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:19.868 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:19.923 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-16T07:48:19.955 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-16T07:48:20.047 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-16T07:48:20.083 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-16T07:48:20.260 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-16T07:48:20.290 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-16T07:48:20.366 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-16T07:48:20.379 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-16T07:48:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-16T07:48:20.641 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-16T07:48:20.670 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-16T07:48:20.712 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-16T07:48:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-16T07:48:20.906 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-16T07:48:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-16T07:48:21.011 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-16T07:48:21.118 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-16T07:48:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-16T07:48:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-16T07:48:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-16T07:48:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-16T07:48:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 13/13 2024-04-16T07:48:22.338 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 13/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 1/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-16T07:48:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-16T07:48:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-16T07:48:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-16T07:48:22.981 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-16T07:48:22.981 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:22.981 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:22.981 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:22.981 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T07:48:22.982 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T07:48:22.982 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T07:48:22.982 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T07:48:22.982 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T07:48:22.982 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T07:48:22.982 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T07:48:22.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T07:48:22.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T07:48:22.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T07:48:22.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T07:48:22.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T07:48:22.983 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:22.983 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:23.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-cephadm 2024-04-16T07:48:23.362 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-16T07:48:23.388 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0a 12/12 2024-04-16T07:48:23.805 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:24 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:23.958 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:23.958 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================== 2024-04-16T07:48:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:48:23.959 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================== 2024-04-16T07:48:23.959 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 142 k 2024-04-16T07:48:23.959 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:48:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T07:48:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T07:48:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T07:48:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T07:48:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T07:48:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:23.961 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:23.961 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================== 2024-04-16T07:48:23.961 INFO:teuthology.orchestra.run.smithi164.stdout:Install 6 Packages 2024-04-16T07:48:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:23.962 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 6.8 M 2024-04-16T07:48:23.962 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 33 M 2024-04-16T07:48:23.963 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:24.604 INFO:teuthology.orchestra.run.smithi164.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-04-16T07:48:24.696 INFO:teuthology.orchestra.run.smithi164.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-04-16T07:48:24.829 INFO:teuthology.orchestra.run.smithi164.stdout:(3/6): ceph-mgr-cephadm-19.0.0-3109.g30b0aa00.e 359 kB/s | 142 kB 00:00 2024-04-16T07:48:24.863 INFO:teuthology.orchestra.run.smithi164.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 346 kB/s | 58 kB 00:00 2024-04-16T07:48:24.997 INFO:teuthology.orchestra.run.smithi164.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.4 MB/s | 548 kB 00:00 2024-04-16T07:48:25.022 INFO:teuthology.orchestra.run.smithi164.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 265 kB/s | 51 kB 00:00 2024-04-16T07:48:25.023 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:25.024 INFO:teuthology.orchestra.run.smithi164.stdout:Total 6.4 MB/s | 6.8 MB 00:01 2024-04-16T07:48:25.037 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:25.047 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:25.047 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:25.137 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:25.138 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:25.443 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:25.617 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-16T07:48:25.666 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-16T07:48:25.846 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0a 12/12 2024-04-16T07:48:25.846 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0a 1/12 2024-04-16T07:48:25.846 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-16T07:48:25.846 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-16T07:48:25.846 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-16T07:48:25.847 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-16T07:48:25.847 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-16T07:48:25.847 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-16T07:48:25.847 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-16T07:48:25.847 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-16T07:48:25.847 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-16T07:48:25.847 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-16T07:48:26.211 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T07:48:26.285 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T07:48:26.286 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T07:48:26.286 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T07:48:26.286 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T07:48:26.286 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T07:48:26.286 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T07:48:26.286 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:26.286 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-16T07:48:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-16T07:48:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 6/6 2024-04-16T07:48:26.704 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-rook 2024-04-16T07:48:27.139 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 6/6 2024-04-16T07:48:27.139 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/6 2024-04-16T07:48:27.139 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-16T07:48:27.140 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-16T07:48:27.140 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-16T07:48:27.140 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-16T07:48:27.293 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:24 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:27.448 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:27.449 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================== 2024-04-16T07:48:27.449 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:48:27.449 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================== 2024-04-16T07:48:27.449 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:27.449 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 49 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T07:48:27.450 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:27.451 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:27.451 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================== 2024-04-16T07:48:27.451 INFO:teuthology.orchestra.run.smithi130.stdout:Install 13 Packages 2024-04-16T07:48:27.451 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:27.451 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 2.0 M 2024-04-16T07:48:27.451 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 26 M 2024-04-16T07:48:27.451 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:27.479 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:27.781 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-fuse 2024-04-16T07:48:27.929 INFO:teuthology.orchestra.run.smithi130.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 205 kB/s | 24 kB 00:00 2024-04-16T07:48:27.954 INFO:teuthology.orchestra.run.smithi130.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 125 kB/s | 18 kB 00:00 2024-04-16T07:48:28.021 INFO:teuthology.orchestra.run.smithi130.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 174 kB 00:00 2024-04-16T07:48:28.038 INFO:teuthology.orchestra.run.smithi130.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-04-16T07:48:28.064 INFO:teuthology.orchestra.run.smithi130.stdout:(5/13): ceph-mgr-rook-19.0.0-3109.g30b0aa00.el9 194 kB/s | 49 kB 00:00 2024-04-16T07:48:28.089 INFO:teuthology.orchestra.run.smithi130.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.1 MB/s | 211 kB 00:00 2024-04-16T07:48:28.114 INFO:teuthology.orchestra.run.smithi130.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-04-16T07:48:28.362 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:29 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:28.517 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:28.518 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:48:28.518 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:28.518 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 871 k 2024-04-16T07:48:28.518 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:48:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:Install 2 Packages 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 950 k 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 4.0 M 2024-04-16T07:48:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:28.523 INFO:teuthology.orchestra.run.smithi130.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 35 kB/s | 15 kB 00:00 2024-04-16T07:48:28.548 INFO:teuthology.orchestra.run.smithi130.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 70 kB/s | 32 kB 00:00 2024-04-16T07:48:28.649 INFO:teuthology.orchestra.run.smithi164.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 611 kB/s | 79 kB 00:00 2024-04-16T07:48:28.831 INFO:teuthology.orchestra.run.smithi130.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 283 kB/s | 203 kB 00:00 2024-04-16T07:48:28.848 INFO:teuthology.orchestra.run.smithi130.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 197 kB/s | 59 kB 00:00 2024-04-16T07:48:29.024 INFO:teuthology.orchestra.run.smithi130.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 467 kB/s | 90 kB 00:00 2024-04-16T07:48:29.074 INFO:teuthology.orchestra.run.smithi130.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-04-16T07:48:29.078 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:29.079 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-04-16T07:48:29.079 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:29.092 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:29.092 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:29.174 INFO:teuthology.orchestra.run.smithi164.stdout:(2/2): ceph-fuse-19.0.0-3109.g30b0aa00.el9.x86_ 1.3 MB/s | 871 kB 00:00 2024-04-16T07:48:29.175 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:29.176 INFO:teuthology.orchestra.run.smithi164.stdout:Total 1.4 MB/s | 950 kB 00:00 2024-04-16T07:48:29.176 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:29.182 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:29.182 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:29.197 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:29.197 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:29.271 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:29.272 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:29.456 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:29.610 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-16T07:48:29.676 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/2 2024-04-16T07:48:29.688 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:29.861 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-16T07:48:29.916 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-16T07:48:29.971 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-16T07:48:30.009 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-16T07:48:30.050 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-16T07:48:30.143 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-16T07:48:30.252 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-16T07:48:30.302 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-16T07:48:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/2 2024-04-16T07:48:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/2 2024-04-16T07:48:30.687 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-16T07:48:30.687 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:30.687 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:30.688 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-16T07:48:30.688 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:30.688 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:30.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-volume 2024-04-16T07:48:31.017 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-16T07:48:31.065 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-16T07:48:31.099 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-16T07:48:31.147 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-16T07:48:31.169 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 13/13 2024-04-16T07:48:31.544 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:32 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:31.700 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 273 k 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:31.701 INFO:teuthology.orchestra.run.smithi164.stdout:Install 2 Packages 2024-04-16T07:48:31.702 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:31.702 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 342 k 2024-04-16T07:48:31.702 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 1.5 M 2024-04-16T07:48:31.702 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:31.822 INFO:teuthology.orchestra.run.smithi164.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 573 kB/s | 69 kB 00:00 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 13/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 1/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-16T07:48:31.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-16T07:48:31.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-16T07:48:31.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-16T07:48:32.081 INFO:teuthology.orchestra.run.smithi164.stdout:(2/2): ceph-volume-19.0.0-3109.g30b0aa00.el9.no 721 kB/s | 273 kB 00:00 2024-04-16T07:48:32.082 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:32.082 INFO:teuthology.orchestra.run.smithi164.stdout:Total 897 kB/s | 342 kB 00:00 2024-04-16T07:48:32.083 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:32.091 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:32.091 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:32.133 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:32.134 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:32.338 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-16T07:48:32.338 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:32.338 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:32.338 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:32.338 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:32.339 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:32.537 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-16T07:48:32.567 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 2/2 2024-04-16T07:48:32.574 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-cephadm 2024-04-16T07:48:32.584 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 2/2 2024-04-16T07:48:32.584 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:48:32.584 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T07:48:32.584 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:33.159 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:30 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:33.313 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================== 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================== 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 142 k 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T07:48:33.314 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T07:48:33.315 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T07:48:33.315 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:33.315 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:33.315 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================== 2024-04-16T07:48:33.315 INFO:teuthology.orchestra.run.smithi130.stdout:Install 6 Packages 2024-04-16T07:48:33.315 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:33.315 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 6.8 M 2024-04-16T07:48:33.316 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 33 M 2024-04-16T07:48:33.316 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 1/2 2024-04-16T07:48:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-16T07:48:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:33.671 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T07:48:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:33.671 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:33.691 INFO:teuthology.orchestra.run.smithi130.stdout:(1/6): ceph-mgr-cephadm-19.0.0-3109.g30b0aa00.e 650 kB/s | 142 kB 00:00 2024-04-16T07:48:33.766 INFO:teuthology.orchestra.run.smithi130.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-04-16T07:48:33.883 INFO:teuthology.orchestra.run.smithi130.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 604 kB/s | 248 kB 00:00 2024-04-16T07:48:33.942 INFO:teuthology.orchestra.run.smithi130.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 329 kB/s | 58 kB 00:00 2024-04-16T07:48:33.967 INFO:teuthology.orchestra.run.smithi130.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 612 kB/s | 51 kB 00:00 2024-04-16T07:48:33.979 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados-devel 2024-04-16T07:48:34.017 INFO:teuthology.orchestra.run.smithi130.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.7 MB/s | 548 kB 00:00 2024-04-16T07:48:34.019 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:34.019 INFO:teuthology.orchestra.run.smithi130.stdout:Total 9.7 MB/s | 6.8 MB 00:00 2024-04-16T07:48:34.033 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:34.043 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:34.043 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:34.130 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:34.131 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:34.426 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:34.579 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:35 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:34.589 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-16T07:48:34.635 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-16T07:48:34.730 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:34.730 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 125 k 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 125 k 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 445 k 2024-04-16T07:48:34.731 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:35.030 INFO:teuthology.orchestra.run.smithi164.stdout:librados-devel-19.0.0-3109.g30b0aa00.el9.x86_64 419 kB/s | 125 kB 00:00 2024-04-16T07:48:35.031 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:35.031 INFO:teuthology.orchestra.run.smithi164.stdout:Total 417 kB/s | 125 kB 00:00 2024-04-16T07:48:35.031 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:35.036 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:35.036 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:35.092 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:35.093 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:35.153 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-16T07:48:35.231 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:35.237 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-16T07:48:35.301 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-16T07:48:35.330 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 6/6 2024-04-16T07:48:35.337 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:36.116 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:36.116 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:36.124 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 6/6 2024-04-16T07:48:36.125 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 1/6 2024-04-16T07:48:36.125 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-16T07:48:36.125 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-16T07:48:36.125 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-16T07:48:36.125 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-16T07:48:36.343 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs2 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:36.419 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:36.676 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-fuse 2024-04-16T07:48:36.948 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:37 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:37.052 INFO:teuthology.orchestra.run.smithi164.stdout:Package libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:37.105 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:37.106 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:48:37.106 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:37.163 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs-devel 2024-04-16T07:48:37.260 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:34 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:37.415 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 871 k 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout:Install 2 Packages 2024-04-16T07:48:37.416 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:37.417 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 950 k 2024-04-16T07:48:37.417 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 4.0 M 2024-04-16T07:48:37.417 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:37.543 INFO:teuthology.orchestra.run.smithi130.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 629 kB/s | 79 kB 00:00 2024-04-16T07:48:37.750 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:38 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:37.901 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 32 k 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-04-16T07:48:37.902 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:37.903 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 32 k 2024-04-16T07:48:37.903 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 142 k 2024-04-16T07:48:37.903 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:38.076 INFO:teuthology.orchestra.run.smithi164.stdout:libcephfs-devel-19.0.0-3109.g30b0aa00.el9.x86_6 183 kB/s | 32 kB 00:00 2024-04-16T07:48:38.077 INFO:teuthology.orchestra.run.smithi130.stdout:(2/2): ceph-fuse-19.0.0-3109.g30b0aa00.el9.x86_ 1.3 MB/s | 871 kB 00:00 2024-04-16T07:48:38.077 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:38.077 INFO:teuthology.orchestra.run.smithi164.stdout:Total 182 kB/s | 32 kB 00:00 2024-04-16T07:48:38.077 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:38.077 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:38.078 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.4 MB/s | 950 kB 00:00 2024-04-16T07:48:38.078 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:38.080 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:38.080 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:38.102 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:38.102 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:38.117 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:38.117 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:38.175 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:38.176 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:38.358 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:38.406 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:38.552 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-16T07:48:38.618 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/2 2024-04-16T07:48:38.725 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:38.985 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:38.986 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:38.986 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:38.986 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:38.986 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:38.986 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:39.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados2 2024-04-16T07:48:39.315 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2/2 2024-04-16T07:48:39.316 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/2 2024-04-16T07:48:39.615 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-16T07:48:39.615 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:39.615 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:39.615 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-16T07:48:39.615 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:39.615 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:39.764 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:40 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:39.856 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-volume 2024-04-16T07:48:39.862 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:39.915 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:39.916 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:48:39.916 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:39.982 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librbd1 2024-04-16T07:48:40.449 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:37 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:40.559 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:41 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:40.606 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:40.607 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:40.607 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-16T07:48:40.607 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:40.607 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:40.607 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:19.0.0-3109.g30b0aa00.el9 ceph-noarch 273 k 2024-04-16T07:48:40.607 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-16T07:48:40.607 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T07:48:40.608 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:40.608 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:40.608 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:40.608 INFO:teuthology.orchestra.run.smithi130.stdout:Install 2 Packages 2024-04-16T07:48:40.608 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:40.608 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 342 k 2024-04-16T07:48:40.609 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 1.5 M 2024-04-16T07:48:40.609 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:40.659 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:40.711 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:40.713 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:48:40.713 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:40.732 INFO:teuthology.orchestra.run.smithi130.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 557 kB/s | 69 kB 00:00 2024-04-16T07:48:40.770 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rados 2024-04-16T07:48:41.107 INFO:teuthology.orchestra.run.smithi130.stdout:(2/2): ceph-volume-19.0.0-3109.g30b0aa00.el9.no 548 kB/s | 273 kB 00:00 2024-04-16T07:48:41.108 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:41.109 INFO:teuthology.orchestra.run.smithi130.stdout:Total 682 kB/s | 342 kB 00:00 2024-04-16T07:48:41.109 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:41.118 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:41.118 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:41.156 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:41.157 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:41.353 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:42 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:41.367 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:41.453 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:41.504 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:41.506 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:48:41.506 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:41.543 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-16T07:48:41.570 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rgw 2024-04-16T07:48:41.593 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 2/2 2024-04-16T07:48:41.609 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 2/2 2024-04-16T07:48:41.610 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:48:41.610 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T07:48:41.610 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:42.155 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:43 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:48:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:42.354 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 1/2 2024-04-16T07:48:42.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-cephfs 2024-04-16T07:48:42.718 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-16T07:48:42.718 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:42.718 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:42.718 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:19.0.0-3109.g30b0aa00.el9.noarch 2024-04-16T07:48:42.719 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T07:48:42.719 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:42.719 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:42.955 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:43 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:42.996 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install librados-devel 2024-04-16T07:48:43.054 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:43.106 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:43.107 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:48:43.107 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:43.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rbd 2024-04-16T07:48:43.617 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:40 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:43.754 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:44 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:43.772 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repo Size 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 125 k 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-04-16T07:48:43.773 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:43.774 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 125 k 2024-04-16T07:48:43.774 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 445 k 2024-04-16T07:48:43.774 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:43.855 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:43.908 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:43.909 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-16T07:48:43.909 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:43.973 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-fuse 2024-04-16T07:48:44.088 INFO:teuthology.orchestra.run.smithi130.stdout:librados-devel-19.0.0-3109.g30b0aa00.el9.x86_64 398 kB/s | 125 kB 00:00 2024-04-16T07:48:44.088 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:44.089 INFO:teuthology.orchestra.run.smithi130.stdout:Total 396 kB/s | 125 kB 00:00 2024-04-16T07:48:44.089 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:44.094 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:44.095 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:44.150 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:44.150 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:44.295 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:44.396 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:44.556 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:45 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:44.710 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 90 k 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 90 k 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 313 k 2024-04-16T07:48:44.711 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:44.901 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:44.912 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-fuse-19.0.0-3109.g30b0aa00.el9.x86_64.rpm 446 kB/s | 90 kB 00:00 2024-04-16T07:48:44.912 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:44.913 INFO:teuthology.orchestra.run.smithi164.stdout:Total 444 kB/s | 90 kB 00:00 2024-04-16T07:48:44.913 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:44.922 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:44.922 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:44.978 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:44.978 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:45.136 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:45.186 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:45.186 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:45.186 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:45.187 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:45.187 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:45.187 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:45.244 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:45.425 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install libcephfs2 2024-04-16T07:48:45.638 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:45.945 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:45.945 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:46.017 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:43 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:46.120 INFO:teuthology.orchestra.run.smithi130.stdout:Package libcephfs2-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:46.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-mirror 2024-04-16T07:48:46.172 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:46.173 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:46.173 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:46.240 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install libcephfs-devel 2024-04-16T07:48:46.782 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:47 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:46.824 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:43 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:46.938 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 3.4 M 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 3.4 M 2024-04-16T07:48:46.942 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 29 M 2024-04-16T07:48:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:46.975 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:46.976 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:46.976 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repo Size 2024-04-16T07:48:46.976 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:46.976 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:46.976 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 32 k 2024-04-16T07:48:46.976 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:46.977 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:46.977 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:46.977 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-04-16T07:48:46.977 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:46.977 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 32 k 2024-04-16T07:48:46.977 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 142 k 2024-04-16T07:48:46.977 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:47.208 INFO:teuthology.orchestra.run.smithi130.stdout:libcephfs-devel-19.0.0-3109.g30b0aa00.el9.x86_6 137 kB/s | 32 kB 00:00 2024-04-16T07:48:47.208 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:47.208 INFO:teuthology.orchestra.run.smithi130.stdout:Total 137 kB/s | 32 kB 00:00 2024-04-16T07:48:47.209 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:47.212 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:47.212 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:47.234 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:47.234 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:47.327 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:47.439 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:47.749 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-mirror-19.0.0-3109.g30b0aa00.el9.x86_64.rpm 4.2 MB/s | 3.4 MB 00:00 2024-04-16T07:48:47.749 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:47.750 INFO:teuthology.orchestra.run.smithi164.stdout:Total 4.2 MB/s | 3.4 MB 00:00 2024-04-16T07:48:47.750 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:47.761 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:47.761 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:47.837 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:47.838 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:47.841 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:48.096 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:48.096 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:48.097 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:48.097 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:48.097 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:48.097 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:48.267 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install librados2 2024-04-16T07:48:48.396 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:48.467 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:48.493 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:48.493 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:48:48.493 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T07:48:48.493 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T07:48:48.493 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T07:48:48.493 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:48.850 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:45 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:48.951 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:49.003 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:49.004 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:49.004 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:49.064 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install librbd1 2024-04-16T07:48:49.648 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:46 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:49.735 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:49.736 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:49.748 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:49.804 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:49.804 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:49.804 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:49.864 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-rados 2024-04-16T07:48:49.976 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-nbd 2024-04-16T07:48:50.447 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:47 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:50.548 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-rados-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:50.568 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:06:51 ago on Tue 16 Apr 2024 07:41:59 AM UTC. 2024-04-16T07:48:50.602 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:50.603 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:50.603 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:50.664 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-rgw 2024-04-16T07:48:50.721 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-16T07:48:50.721 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-04-16T07:48:50.721 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:50.721 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-16T07:48:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 175 k 2024-04-16T07:48:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:50.722 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-16T07:48:50.722 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-16T07:48:50.722 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-04-16T07:48:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:50.722 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 175 k 2024-04-16T07:48:50.722 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 673 k 2024-04-16T07:48:50.722 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-16T07:48:51.061 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-nbd-19.0.0-3109.g30b0aa00.el9.x86_64.rpm 518 kB/s | 175 kB 00:00 2024-04-16T07:48:51.062 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:51.062 INFO:teuthology.orchestra.run.smithi164.stdout:Total 516 kB/s | 175 kB 00:00 2024-04-16T07:48:51.062 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-16T07:48:51.072 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-16T07:48:51.072 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-16T07:48:51.128 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-16T07:48:51.128 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-16T07:48:51.242 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:48 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-16T07:48:51.342 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-rgw-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:51.377 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:51.395 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:51.396 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:51.396 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:51.455 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-cephfs 2024-04-16T07:48:51.807 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:52.035 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:49 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:52.101 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:52.101 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:52.101 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-16T07:48:52.101 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:52.101 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:48:52.101 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-16T07:48:52.132 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-cephfs-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:52.185 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:52.186 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:52.186 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:52.250 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-rbd 2024-04-16T07:48:52.348 DEBUG:teuthology.parallel:result is None 2024-04-16T07:48:52.835 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:49 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:52.935 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-rbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 is already installed. 2024-04-16T07:48:52.989 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:52.990 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-16T07:48:52.990 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:53.051 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install rbd-fuse 2024-04-16T07:48:53.632 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:50 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:53.784 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 90 k 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-04-16T07:48:53.785 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:53.786 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 90 k 2024-04-16T07:48:53.786 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 313 k 2024-04-16T07:48:53.786 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:48:53.969 INFO:teuthology.orchestra.run.smithi130.stdout:rbd-fuse-19.0.0-3109.g30b0aa00.el9.x86_64.rpm 491 kB/s | 90 kB 00:00 2024-04-16T07:48:53.969 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:48:53.970 INFO:teuthology.orchestra.run.smithi130.stdout:Total 487 kB/s | 90 kB 00:00 2024-04-16T07:48:53.970 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:48:53.978 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:48:53.979 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:48:54.033 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:48:54.033 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:48:54.180 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:48:54.284 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:54.685 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:54.992 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:48:54.992 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:54.992 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:48:54.992 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:48:54.992 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:54.992 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:48:55.220 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install rbd-mirror 2024-04-16T07:48:55.814 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:06:52 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:48:55.969 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repo Size 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 3.4 M 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 3.4 M 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 29 M 2024-04-16T07:48:55.970 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:49:28.092 INFO:teuthology.orchestra.run.smithi130.stdout:rbd-mirror-19.0.0-3109.g30b0aa00.el9.x86_64.rpm 108 kB/s | 3.4 MB 00:32 2024-04-16T07:49:28.093 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:49:28.093 INFO:teuthology.orchestra.run.smithi130.stdout:Total 108 kB/s | 3.4 MB 00:32 2024-04-16T07:49:28.093 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:49:28.106 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:49:28.106 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:49:28.184 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:49:28.185 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:49:28.746 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:49:28.842 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:49:28.868 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:49:28.868 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T07:49:28.868 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T07:49:28.868 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T07:49:28.869 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T07:49:28.869 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:30.143 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:49:30.144 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:30.144 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:49:30.144 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:49:30.144 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:30.144 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:49:30.379 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install rbd-nbd 2024-04-16T07:49:30.977 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:07:27 ago on Tue 16 Apr 2024 07:42:03 AM UTC. 2024-04-16T07:49:31.129 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:19.0.0-3109.g30b0aa00.el9 ceph 175 k 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-04-16T07:49:31.130 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:31.131 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 175 k 2024-04-16T07:49:31.131 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 673 k 2024-04-16T07:49:31.131 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-16T07:49:31.619 INFO:teuthology.orchestra.run.smithi130.stdout:rbd-nbd-19.0.0-3109.g30b0aa00.el9.x86_64.rpm 359 kB/s | 175 kB 00:00 2024-04-16T07:49:31.620 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-16T07:49:31.620 INFO:teuthology.orchestra.run.smithi130.stdout:Total 358 kB/s | 175 kB 00:00 2024-04-16T07:49:31.620 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-16T07:49:31.630 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-16T07:49:31.630 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-16T07:49:31.687 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-16T07:49:31.687 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-16T07:49:31.836 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-16T07:49:31.938 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:49:32.386 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:49:32.677 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 1/1 2024-04-16T07:49:32.677 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:32.677 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-16T07:49:32.677 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:19.0.0-3109.g30b0aa00.el9.x86_64 2024-04-16T07:49:32.677 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:32.677 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-16T07:49:32.912 DEBUG:teuthology.parallel:result is None 2024-04-16T07:49:32.912 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=30b0aa0009a925d6a46536e55c5656edcd51ecdb 2024-04-16T07:49:33.189 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T07:49:33.211 INFO:teuthology.orchestra.run.smithi130.stdout:19.0.0-3109.g30b0aa00.el9 2024-04-16T07:49:33.211 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3109.g30b0aa00.el9 2024-04-16T07:49:33.212 INFO:teuthology.task.install:The correct ceph version 19.0.0-3109.g30b0aa00 is installed. 2024-04-16T07:49:33.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=30b0aa0009a925d6a46536e55c5656edcd51ecdb 2024-04-16T07:49:33.492 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T07:49:33.518 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3109.g30b0aa00.el9 2024-04-16T07:49:33.518 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3109.g30b0aa00.el9 2024-04-16T07:49:33.518 INFO:teuthology.task.install:The correct ceph version 19.0.0-3109.g30b0aa00 is installed. 2024-04-16T07:49:33.520 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-16T07:49:33.520 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:33.520 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T07:49:33.550 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:33.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T07:49:33.592 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-16T07:49:33.593 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:33.593 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T07:49:33.654 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T07:49:33.723 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:33.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T07:49:33.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T07:49:33.825 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-16T07:49:33.825 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:33.826 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T07:49:33.853 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T07:49:33.921 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:33.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T07:49:33.952 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T07:49:34.021 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-16T07:49:34.022 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:34.022 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T07:49:34.050 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T07:49:34.118 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:34.118 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T07:49:34.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T07:49:34.216 INFO:teuthology.run_tasks:Running task ceph... 2024-04-16T07:49:34.311 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-16T07:49:34.311 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /var/log/ceph 2024-04-16T07:49:34.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-04-16T07:49:34.344 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-16T07:49:34.344 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T07:49:34.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T07:49:34.413 INFO:tasks.ceph:Creating extra log directories... 2024-04-16T07:49:34.413 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T07:49:34.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T07:49:34.489 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-16T07:49:34.490 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-16T07:49:34.490 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979', 'branch': 'wip-matanb-crimson-replicated-april-15', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7657979', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-replicated-april-15'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'a7639cb6ce633dabc4d37769d62caacb54582984'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 6444, 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'a7639cb6ce633dabc4d37769d62caacb54582984', 'targets': {'smithi130.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ3frryKjjuLRnl70qaclrLNafzBX76iNXxFw3R1ss5XsTmwjUsliogWn4rfLFlamMR2t9AaHgOzLQU2HkbJd8s=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCAVTSCYMgv1PQYlBnurqYd7NhPHCnsHcIcI6lKQiTWoqpmW7iVCioIXHo5yYKnvin+Ne7wIGNkWxplxHDc58s='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '30b0aa0009a925d6a46536e55c5656edcd51ecdb', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-16_07:26:08', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-16T07:49:34.490 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-16T07:49:34.520 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-16T07:49:34.538 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T07:49:34.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T07:49:34.607 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:34.607 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T07:49:34.656 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T07:49:34.656 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-16 07:49:29.111278937 +0000 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-16 07:40:52.987897427 +0000 2024-04-16T07:49:34.712 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-16 07:40:52.987897427 +0000 2024-04-16T07:49:34.713 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-16 07:40:52.987897427 +0000 2024-04-16T07:49:34.713 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T07:49:34.778 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-16T07:49:34.779 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-16T07:49:34.779 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000205519 s, 2.5 MB/s 2024-04-16T07:49:34.780 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T07:49:34.836 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-04-16T07:49:34.891 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T07:49:34.891 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:34.892 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-16T07:49:34.892 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:34.892 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:34.892 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-16 07:49:29.112278918 +0000 2024-04-16T07:49:34.892 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-16 07:40:53.333890690 +0000 2024-04-16T07:49:34.892 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-16 07:40:53.333890690 +0000 2024-04-16T07:49:34.892 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-16 07:40:53.333890690 +0000 2024-04-16T07:49:34.892 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T07:49:34.959 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-16T07:49:34.959 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-16T07:49:34.959 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000188007 s, 2.7 MB/s 2024-04-16T07:49:34.960 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T07:49:35.017 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-16 07:49:29.112278918 +0000 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-16 07:40:53.683883874 +0000 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-16 07:40:53.683883874 +0000 2024-04-16T07:49:35.073 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-16 07:40:53.683883874 +0000 2024-04-16T07:49:35.074 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T07:49:35.140 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-16T07:49:35.140 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-16T07:49:35.140 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000212892 s, 2.4 MB/s 2024-04-16T07:49:35.141 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T07:49:35.197 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-16 07:49:29.113278898 +0000 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-16 07:40:54.009877525 +0000 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-16 07:40:54.009877525 +0000 2024-04-16T07:49:35.253 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-16 07:40:54.009877525 +0000 2024-04-16T07:49:35.254 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T07:49:35.320 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-16T07:49:35.320 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-16T07:49:35.320 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000194521 s, 2.6 MB/s 2024-04-16T07:49:35.321 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T07:49:35.378 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-16T07:49:35.378 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:35.378 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T07:49:35.394 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T07:49:35.395 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-16 07:48:48.743805049 +0000 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-16 07:40:52.990971867 +0000 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-16 07:40:52.990971867 +0000 2024-04-16T07:49:35.451 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-16 07:40:52.990971867 +0000 2024-04-16T07:49:35.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T07:49:35.518 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-16T07:49:35.518 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-16T07:49:35.518 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000188783 s, 2.7 MB/s 2024-04-16T07:49:35.519 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T07:49:35.578 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-04-16T07:49:35.633 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T07:49:35.633 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:35.633 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-16T07:49:35.633 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:35.633 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:35.633 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-16 07:48:48.743805049 +0000 2024-04-16T07:49:35.634 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-16 07:40:53.336965398 +0000 2024-04-16T07:49:35.634 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-16 07:40:53.336965398 +0000 2024-04-16T07:49:35.634 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-16 07:40:53.336965398 +0000 2024-04-16T07:49:35.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T07:49:35.699 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-16T07:49:35.700 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-16T07:49:35.700 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000219399 s, 2.3 MB/s 2024-04-16T07:49:35.701 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T07:49:35.758 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-04-16T07:49:35.813 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T07:49:35.813 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:35.814 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-16T07:49:35.814 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:35.814 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:35.814 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-16 07:48:48.744805031 +0000 2024-04-16T07:49:35.814 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-16 07:40:53.665959245 +0000 2024-04-16T07:49:35.814 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-16 07:40:53.665959245 +0000 2024-04-16T07:49:35.814 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-16 07:40:53.665959245 +0000 2024-04-16T07:49:35.814 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T07:49:35.880 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-16T07:49:35.881 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-16T07:49:35.881 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00022588 s, 2.3 MB/s 2024-04-16T07:49:35.882 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T07:49:35.938 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-04-16T07:49:35.994 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T07:49:35.994 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T07:49:35.995 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-16T07:49:35.995 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T07:49:35.995 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T07:49:35.995 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-16 07:48:48.744805031 +0000 2024-04-16T07:49:35.995 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-16 07:40:54.014952720 +0000 2024-04-16T07:49:35.995 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-16 07:40:54.014952720 +0000 2024-04-16T07:49:35.995 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-16 07:40:54.014952720 +0000 2024-04-16T07:49:35.995 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T07:49:36.061 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-16T07:49:36.061 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-16T07:49:36.062 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000207689 s, 2.5 MB/s 2024-04-16T07:49:36.063 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T07:49:36.119 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-16T07:49:36.119 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi130.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-04-16T07:49:36.119 INFO:tasks.ceph:Generating config... 2024-04-16T07:49:36.120 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-04-16T07:49:36.120 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-04-16T07:49:36.120 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-04-16T07:49:36.121 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-04-16T07:49:36.121 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-04-16T07:49:36.121 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-16T07:49:36.121 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-16T07:49:36.121 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-16T07:49:36.121 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-04-16T07:49:36.121 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-04-16T07:49:36.121 INFO:tasks.ceph:[osd] debug monc = 20 2024-04-16T07:49:36.121 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-04-16T07:49:36.121 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-04-16T07:49:36.121 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-16T07:49:36.121 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-16T07:49:36.122 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-16T07:49:36.122 INFO:tasks.ceph:[osd] osd class default list = * 2024-04-16T07:49:36.122 INFO:tasks.ceph:[osd] osd class load list = * 2024-04-16T07:49:36.122 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-16T07:49:36.122 INFO:tasks.ceph:[client] debug ms = 1 2024-04-16T07:49:36.122 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-04-16T07:49:36.122 INFO:tasks.ceph:[global] ms client mode = crc 2024-04-16T07:49:36.122 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-04-16T07:49:36.122 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-04-16T07:49:36.122 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-04-16T07:49:36.122 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-04-16T07:49:36.122 INFO:tasks.ceph:[global] ms service mode = crc 2024-04-16T07:49:36.123 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-16T07:49:36.123 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-16T07:49:36.123 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-04-16T07:49:36.123 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-04-16T07:49:36.123 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-04-16T07:49:36.123 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-04-16T07:49:36.123 INFO:tasks.ceph:Setting up mon.a... 2024-04-16T07:49:36.123 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-16T07:49:36.190 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/ceph.keyring 2024-04-16T07:49:36.194 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-16T07:49:36.251 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-16T07:49:36.280 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.130')] 2024-04-16T07:49:36.280 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.130', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-04-16T07:49:36.281 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:36.281 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T07:49:36.336 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.130 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:36.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:36.427+0000 7fdafb22df00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:36.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:36.427+0000 7fdafb22df00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:36.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:36.427+0000 7fdafb22df00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:monmaptool: generated fsid 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:setting min_mon_release = quincy 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:epoch 0 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:fsid 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:last_changed 2024-04-16T07:49:36.428919+0000 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:created 2024-04-16T07:49:36.428919+0000 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:min_mon_release 17 (quincy) 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:election_strategy: 1 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:0: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.a 2024-04-16T07:49:36.428 INFO:teuthology.orchestra.run.smithi130.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-04-16T07:49:36.429 DEBUG:teuthology.orchestra.run.smithi130:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T07:49:36.445 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 06ce95cb-daea-4a00-b123-d08aef1a06ab... 2024-04-16T07:49:36.445 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-16T07:49:36.488 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-16T07:49:36.538 INFO:teuthology.orchestra.run.smithi130.stdout:[global] 2024-04-16T07:49:36.538 INFO:teuthology.orchestra.run.smithi130.stdout: chdir = "" 2024-04-16T07:49:36.538 INFO:teuthology.orchestra.run.smithi130.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T07:49:36.538 INFO:teuthology.orchestra.run.smithi130.stdout: auth supported = cephx 2024-04-16T07:49:36.538 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.538 INFO:teuthology.orchestra.run.smithi130.stdout: filestore xattr use omap = true 2024-04-16T07:49:36.538 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: mon clock drift allowed = 1.000 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: osd crush chooseleaf type = 0 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: auth debug = true 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on old message = true 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on bug = true 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: mon pg warn max object skew = 0 2024-04-16T07:49:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default size = 2 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow primary affinity = true 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow pg remap = true 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on legacy crush tunables = false 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on crush straw calc version zero = false 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on no sortbitwise = false 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on osd down out interval zero = false 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on too few osds = false 2024-04-16T07:49:36.540 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: mon_allow_pool_size_one = true 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: osd default data pool replay window = 5 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: mon allow pool delete = true 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: mon cluster log file level = debug 2024-04-16T07:49:36.541 INFO:teuthology.orchestra.run.smithi130.stdout: debug asserts on shutdown = true 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: mon health detail to clog = false 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: mon host = 172.21.15.130 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: ms client mode = crc 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: ms cluster mode = crc 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: ms mon client mode = crc 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: ms mon cluster mode = crc 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: ms mon service mode = crc 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: ms service mode = crc 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: fsid = 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:36.542 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout:[osd] 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd journal size = 100 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub load threshold = 5.0 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub max interval = 600 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd mclock profile = high_recovery_ops 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd recover clone overlap = true 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd recovery max chunk = 1048576 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug shutdown = true 2024-04-16T07:49:36.543 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug op order = true 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug verify stray on activate = true 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug trim objects = true 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: osd open classes on start = true 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug pg log writeout = true 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: osd deep scrub update digest min age = 30 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: osd map max advance = 10 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: journal zero on create = true 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.544 INFO:teuthology.orchestra.run.smithi130.stdout: filestore ondisk finisher threads = 3 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: filestore apply finisher threads = 3 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: bdev debug aio = true 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug misdirected ops = true 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: debug monc = 20 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: crimson alien thread cpu cores = 6-7 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: crimson osd obc lru size = 10 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: debug osd = 20 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: osd blocked scrub grace period = 3600 2024-04-16T07:49:36.545 INFO:teuthology.orchestra.run.smithi130.stdout: osd class default list = * 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: osd class load list = * 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: osd shutdown pgref assert = True 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr] 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: debug mgr = 20 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2024-04-16T07:49:36.546 INFO:teuthology.orchestra.run.smithi130.stdout: mgr/telemetry/nag = false 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout:[mon] 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: debug paxos = 20 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: mon data avail warn = 5 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: mon mgr mkfs grace = 240 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd reporter subtree level = osd 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd prime pg temp = true 2024-04-16T07:49:36.547 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: # 1m isn't quite enough 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: mon_down_mkfs_grace = 2m 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.548 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_filestore_osds = false 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: mon min osdmap epochs = 50 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: mon osdmap full prune interval = 2 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: mon osdmap full prune min = 15 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: mon osdmap full prune txsize = 2 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: paxos service trim min = 10 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on pool no app = False 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default crimson = True 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout:[client] 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: rgw cache enabled = true 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable ops log = true 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable usage log = true 2024-04-16T07:49:36.549 INFO:teuthology.orchestra.run.smithi130.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.a] 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.0] 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout: crimson seastar cpu cores = 0-2 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.1] 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout: crimson seastar cpu cores = 3-5 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.2] 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout: crimson seastar cpu cores = 0-2 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.3] 2024-04-16T07:49:36.550 INFO:teuthology.orchestra.run.smithi130.stdout: crimson seastar cpu cores = 3-5 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T07:49:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = 172.21.15.130 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: ms client mode = crc 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: ms cluster mode = crc 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: ms mon client mode = crc 2024-04-16T07:49:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: ms mon cluster mode = crc 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: ms mon service mode = crc 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: ms service mode = crc 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-04-16T07:49:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-04-16T07:49:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: debug monc = 20 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: crimson alien thread cpu cores = 6-7 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: crimson osd obc lru size = 10 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-04-16T07:49:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: osd blocked scrub grace period = 3600 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: osd class default list = * 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: osd class load list = * 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: osd shutdown pgref assert = True 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-04-16T07:49:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-04-16T07:49:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon min osdmap epochs = 50 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon osdmap full prune interval = 2 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon osdmap full prune min = 15 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon osdmap full prune txsize = 2 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: paxos service trim min = 10 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on pool no app = False 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default crimson = True 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-04-16T07:49:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: crimson seastar cpu cores = 0-2 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: crimson seastar cpu cores = 3-5 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: crimson seastar cpu cores = 0-2 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-04-16T07:49:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: crimson seastar cpu cores = 3-5 2024-04-16T07:49:36.564 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-16T07:49:36.564 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-16T07:49:36.632 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-16T07:49:36.632 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:36.632 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-16T07:49:36.648 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:36.648 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-16T07:49:36.703 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi130.front.sepia.ceph.com 2024-04-16T07:49:36.704 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:36.704 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-16T07:49:36.704 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-16T07:49:36.784 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:36.785 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:36.840 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-04-16T07:49:36.840 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:36.840 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-16T07:49:36.840 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-16T07:49:36.882 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:36.882 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:36.938 INFO:tasks.ceph:Setting up mon nodes... 2024-04-16T07:49:36.938 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-16T07:49:36.938 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-16T07:49:37.053 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-16T07:49:37.056 INFO:tasks.ceph:Setting up mds nodes... 2024-04-16T07:49:37.056 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-16T07:49:37.056 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-16T07:49:37.108 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-16T07:49:37.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-16T07:49:37.177 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-16T07:49:37.194 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-16T07:49:37.194 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi130.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-04-16T07:49:37.194 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-16T07:49:37.222 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-16T07:49:37.222 INFO:tasks.ceph:role: osd.0 2024-04-16T07:49:37.222 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi130.front.sepia.ceph.com 2024-04-16T07:49:37.222 DEBUG:teuthology.orchestra.run.smithi130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T07:49:37.297 INFO:teuthology.orchestra.run.smithi130.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T07:49:37.297 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout: = sunit=0 swidth=0 blks 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T07:49:37.298 INFO:teuthology.orchestra.run.smithi130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T07:49:37.791 INFO:teuthology.orchestra.run.smithi130.stdout:Discarding blocks...Done. 2024-04-16T07:49:37.792 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi130.front.sepia.ceph.com -o noatime 2024-04-16T07:49:37.793 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-16T07:49:37.917 DEBUG:teuthology.orchestra.run.smithi130:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-16T07:49:37.954 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-16T07:49:38.024 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-16T07:49:38.024 INFO:tasks.ceph:role: osd.1 2024-04-16T07:49:38.024 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi130.front.sepia.ceph.com 2024-04-16T07:49:38.024 DEBUG:teuthology.orchestra.run.smithi130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout: = sunit=0 swidth=0 blks 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T07:49:38.100 INFO:teuthology.orchestra.run.smithi130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T07:49:38.547 INFO:teuthology.orchestra.run.smithi130.stdout:Discarding blocks...Done. 2024-04-16T07:49:38.549 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi130.front.sepia.ceph.com -o noatime 2024-04-16T07:49:38.549 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-16T07:49:38.647 DEBUG:teuthology.orchestra.run.smithi130:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-16T07:49:38.679 DEBUG:teuthology.orchestra.run.smithi130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:39.014 INFO:teuthology.orchestra.run.smithi130.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:39.015 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:39,017 seastar - Reactor backend: linux-aio 2024-04-16T07:49:39.027 INFO:teuthology.orchestra.run.smithi130.stderr:==29121==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:39.145 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:39,147 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-16T07:49:39.145 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:39,147 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-04-16T07:49:39.146 INFO:teuthology.orchestra.run.smithi130.stderr:DEBUG 2024-04-16 07:49:39,148 [shard 0:main] seastar - reactor::drain 2024-04-16T07:49:39.693 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:39.694 INFO:teuthology.orchestra.run.smithi130.stderr:Suppressions used: 2024-04-16T07:49:39.694 INFO:teuthology.orchestra.run.smithi130.stderr: count bytes template 2024-04-16T07:49:39.694 INFO:teuthology.orchestra.run.smithi130.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:39.694 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:39.694 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T07:49:39.752 INFO:teuthology.orchestra.run.smithi130.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:39.756 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:39,757 seastar - Reactor backend: linux-aio 2024-04-16T07:49:39.765 INFO:teuthology.orchestra.run.smithi130.stderr:==29119==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:43.234 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:43.234 INFO:teuthology.orchestra.run.smithi130.stderr:Suppressions used: 2024-04-16T07:49:43.234 INFO:teuthology.orchestra.run.smithi130.stderr: count bytes template 2024-04-16T07:49:43.234 INFO:teuthology.orchestra.run.smithi130.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:43.234 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:43.234 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T07:49:43.234 INFO:teuthology.orchestra.run.smithi130.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:43.260 DEBUG:teuthology.orchestra.run.smithi130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-16T07:49:43.289 DEBUG:teuthology.orchestra.run.smithi130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:43.587 INFO:teuthology.orchestra.run.smithi130.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:43.588 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:43,590 seastar - Reactor backend: linux-aio 2024-04-16T07:49:43.593 INFO:teuthology.orchestra.run.smithi130.stderr:==29929==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:43.713 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:43,714 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-16T07:49:43.713 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:43,714 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-04-16T07:49:43.713 INFO:teuthology.orchestra.run.smithi130.stderr:DEBUG 2024-04-16 07:49:43,715 [shard 0:main] seastar - reactor::drain 2024-04-16T07:49:44.252 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:44.252 INFO:teuthology.orchestra.run.smithi130.stderr:Suppressions used: 2024-04-16T07:49:44.252 INFO:teuthology.orchestra.run.smithi130.stderr: count bytes template 2024-04-16T07:49:44.252 INFO:teuthology.orchestra.run.smithi130.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:44.252 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:44.252 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T07:49:44.310 INFO:teuthology.orchestra.run.smithi130.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:44.314 INFO:teuthology.orchestra.run.smithi130.stderr:INFO 2024-04-16 07:49:44,315 seastar - Reactor backend: linux-aio 2024-04-16T07:49:44.323 INFO:teuthology.orchestra.run.smithi130.stderr:==29927==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:44.422 INFO:teuthology.orchestra.run.smithi130.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb99d759 0xb870220 0xb87179c 0xb8719b2 0xb871b08 0xb871fd1 0x54daf 0x36289 0x368cd39 0x5d8c45d 0x5d8ca56 0x5bd1519 0x5bec5dd 0x6ec6e84 0x6ed8657 0x6ed9af4 0x6ed9b79 0x6ed9c7b 0x6ed9d73 0x6ed9f80 0x6eda166 0x6eda4b5 0x6eda859 0x6edaa89 0x6edacf3 0x6edafc0 0x6edb1de 0x6edb296 0x6edb35c 0x6edb43d 0x5a39e18 0x5a39f22 0x5a39fde 0x5a3a09a 0x5a3a156 0x5a3a2a3 0x5a3a42e 0x5a3a7ef 0x5a56730 0xb882ea5 0xb89d13f 0xb93e13d 0xb93f78b 0xb61bf71 0xb61c8ed 0x367a090 0x3feaf 0x3ff5f 0x3463444 2024-04-16T07:49:44.422 INFO:teuthology.orchestra.run.smithi130.stderr:kernel callstack: 2024-04-16T07:49:47.811 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:47.811 INFO:teuthology.orchestra.run.smithi130.stderr:Suppressions used: 2024-04-16T07:49:47.811 INFO:teuthology.orchestra.run.smithi130.stderr: count bytes template 2024-04-16T07:49:47.811 INFO:teuthology.orchestra.run.smithi130.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:47.811 INFO:teuthology.orchestra.run.smithi130.stderr:----------------------------------------------------- 2024-04-16T07:49:47.811 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T07:49:47.812 INFO:teuthology.orchestra.run.smithi130.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:47.836 DEBUG:teuthology.orchestra.run.smithi130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-16T07:49:47.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-16T07:49:47.899 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-16T07:49:47.899 INFO:tasks.ceph:role: osd.2 2024-04-16T07:49:47.899 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-16T07:49:47.899 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T07:49:47.978 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T07:49:47.979 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T07:49:47.979 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T07:49:48.460 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-16T07:49:48.462 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-16T07:49:48.462 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-16T07:49:48.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-16T07:49:48.618 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-16T07:49:48.689 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-16T07:49:48.689 INFO:tasks.ceph:role: osd.3 2024-04-16T07:49:48.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-16T07:49:48.689 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T07:49:48.766 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T07:49:48.766 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T07:49:48.766 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T07:49:48.766 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T07:49:48.766 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T07:49:48.766 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-16T07:49:48.767 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T07:49:48.767 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T07:49:48.767 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T07:49:48.767 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T07:49:49.229 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-16T07:49:49.231 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-16T07:49:49.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-04-16T07:49:49.323 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-16T07:49:49.359 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:49.704 INFO:teuthology.orchestra.run.smithi164.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:49.705 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:49,706 seastar - Reactor backend: linux-aio 2024-04-16T07:49:49.716 INFO:teuthology.orchestra.run.smithi164.stderr:==28855==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:49.837 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:49,838 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-16T07:49:49.837 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:49,839 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-04-16T07:49:49.838 INFO:teuthology.orchestra.run.smithi164.stderr:DEBUG 2024-04-16 07:49:49,839 [shard 0:main] seastar - reactor::drain 2024-04-16T07:49:50.373 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:50.374 INFO:teuthology.orchestra.run.smithi164.stderr:Suppressions used: 2024-04-16T07:49:50.374 INFO:teuthology.orchestra.run.smithi164.stderr: count bytes template 2024-04-16T07:49:50.374 INFO:teuthology.orchestra.run.smithi164.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:50.374 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:50.374 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-16T07:49:50.430 INFO:teuthology.orchestra.run.smithi164.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:50.434 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:50,435 seastar - Reactor backend: linux-aio 2024-04-16T07:49:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:==28853==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:50.507 INFO:teuthology.orchestra.run.smithi164.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb99d759 0xb870220 0xb87179c 0xb8719b2 0xb871b08 0xb871fd1 0x54daf 0x368ff2d 0x3690322 0x5db1371 0x5db17e6 0x5e0d774 0x5d8c2f0 0x5d8c824 0x5bd1519 0x5bec5dd 0x6ec6e84 0x6ed8657 0x6ed9af4 0x6ed9b79 0x6ed9c7b 0x6ed9d73 0x6ed9f80 0x6eda166 0x6eda4b5 0x6eda859 0x6edaa89 0x6edacf3 0x6edafc0 0x6edb1de 0x6edb296 0x6edb35c 0x6edb43d 0x5a39e18 0x5a39f22 0x5a39fde 0x5a3a09a 0x5a3a156 0x5a3a2a3 0x5a3a42e 0x5a3a7ef 0x5a56730 0xb882ea5 0xb89d13f 0xb93e13d 0xb93f78b 0xb61bf71 0xb61c8ed 0x367a090 0x3feaf 0x3ff5f 0x3463444 2024-04-16T07:49:50.508 INFO:teuthology.orchestra.run.smithi164.stderr:kernel callstack: 2024-04-16T07:49:53.918 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:53.918 INFO:teuthology.orchestra.run.smithi164.stderr:Suppressions used: 2024-04-16T07:49:53.918 INFO:teuthology.orchestra.run.smithi164.stderr: count bytes template 2024-04-16T07:49:53.919 INFO:teuthology.orchestra.run.smithi164.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:53.919 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:53.919 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-16T07:49:53.919 INFO:teuthology.orchestra.run.smithi164.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:53.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-16T07:49:53.975 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:54.272 INFO:teuthology.orchestra.run.smithi164.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:54.273 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:54,274 seastar - Reactor backend: linux-aio 2024-04-16T07:49:54.278 INFO:teuthology.orchestra.run.smithi164.stderr:==29663==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:54.397 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:54,398 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-16T07:49:54.397 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:54,398 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-04-16T07:49:54.398 INFO:teuthology.orchestra.run.smithi164.stderr:DEBUG 2024-04-16 07:49:54,399 [shard 0:main] seastar - reactor::drain 2024-04-16T07:49:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:Suppressions used: 2024-04-16T07:49:54.931 INFO:teuthology.orchestra.run.smithi164.stderr: count bytes template 2024-04-16T07:49:54.931 INFO:teuthology.orchestra.run.smithi164.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:54.932 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-16T07:49:54.991 INFO:teuthology.orchestra.run.smithi164.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-16T07:49:54.995 INFO:teuthology.orchestra.run.smithi164.stderr:INFO 2024-04-16 07:49:54,996 seastar - Reactor backend: linux-aio 2024-04-16T07:49:55.004 INFO:teuthology.orchestra.run.smithi164.stderr:==29661==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-16T07:49:58.473 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:58.474 INFO:teuthology.orchestra.run.smithi164.stderr:Suppressions used: 2024-04-16T07:49:58.474 INFO:teuthology.orchestra.run.smithi164.stderr: count bytes template 2024-04-16T07:49:58.474 INFO:teuthology.orchestra.run.smithi164.stderr: 1 8 MallocExtension::Register 2024-04-16T07:49:58.474 INFO:teuthology.orchestra.run.smithi164.stderr:----------------------------------------------------- 2024-04-16T07:49:58.474 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-16T07:49:58.474 INFO:teuthology.orchestra.run.smithi164.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 06ce95cb-daea-4a00-b123-d08aef1a06ab 2024-04-16T07:49:58.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-16T07:49:58.527 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-16T07:49:58.527 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:58.527 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-16T07:49:58.561 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:58.561 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-16T07:49:58.633 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:58.633 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-16T07:49:58.661 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:58.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-16T07:49:58.729 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:58.729 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-16T07:49:58.797 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:49:58.797 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-16T07:49:58.814 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:49:58.814 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-16T07:49:58.852 INFO:tasks.ceph:Adding keys to all mons... 2024-04-16T07:49:58.853 DEBUG:teuthology.orchestra.run.smithi130:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.0] 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCTLR5miqVPNhAApCpwUmNGF+2EJ0i1us4CMQ== 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.1] 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCYLR5m4y/dHBAAyMCdcyZVOsqocbMaqXreng== 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr.x] 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCRLR5myVlAAxAAUrN5hKmScwzZmFIMZh70Cg== 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.2] 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCeLR5mhAU6IxAAyQ2zP0rDiCwc83T3N03DxQ== 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.3] 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCjLR5mDjcaCRAAD2RpmDDnG+y0/u5WRjomdQ== 2024-04-16T07:49:58.883 INFO:teuthology.orchestra.run.smithi130.stdout:[client.0] 2024-04-16T07:49:58.884 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCRLR5mVoCMBhAA9d1tOmVKWnL9nk6E0PyLkQ== 2024-04-16T07:49:58.884 INFO:teuthology.orchestra.run.smithi130.stdout:[client.1] 2024-04-16T07:49:58.884 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQCRLR5mJ8uoChAAheh/jTJwnZFrIo77Ee7y6g== 2024-04-16T07:49:58.885 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T07:49:58.991 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T07:49:59.055 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-16T07:49:59.118 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T07:49:59.181 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T07:49:59.245 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T07:49:59.311 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T07:49:59.377 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-16T07:49:59.378 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-16T07:49:59.405 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-16T07:49:59.515 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.515+0000 7f5b2462ea80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.515 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.515+0000 7f5b2462ea80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.528 DEBUG:teuthology.orchestra.run.smithi130:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-16T07:49:59.556 DEBUG:teuthology.orchestra.run.smithi130:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-16T07:49:59.611 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-16T07:49:59.612 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-16T07:49:59.612 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-16T07:49:59.653 INFO:tasks.ceph.mon.a:Started 2024-04-16T07:49:59.654 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-16T07:49:59.654 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-16T07:49:59.654 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-16T07:49:59.659 INFO:tasks.ceph.mgr.x:Started 2024-04-16T07:49:59.660 INFO:tasks.ceph:Setting crush tunables to default 2024-04-16T07:49:59.660 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster ceph osd crush tunables default 2024-04-16T07:49:59.739 INFO:tasks.ceph.mon.a.smithi130.stderr:2024-04-16T07:49:59.739+0000 7f944592ca80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.739 INFO:tasks.ceph.mon.a.smithi130.stderr:2024-04-16T07:49:59.739+0000 7f944592ca80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.739 INFO:tasks.ceph.mon.a.smithi130.stderr:2024-04-16T07:49:59.739+0000 7f944592ca80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.762 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:49:59.763+0000 7fafa4a34000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.773 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.768+0000 7f992cb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.775 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.775+0000 7f992cb9d640 1 Processor -- start 2024-04-16T07:49:59.775 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.775+0000 7f992cb9d640 1 -- start start 2024-04-16T07:49:59.776 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f992cb9d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f99280a1f90 0x7f992813a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f992cb9d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f992809eab0 con 0x7f992809f290 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f992cb9d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f992809f950 con 0x7f99280a1f90 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f991ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f99280a1f90 0x7f992813a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f992659c640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f992809f290 0x7f99280a1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:37028/0 (socket says 172.21.15.130:37028) 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f992659c640 1 -- 172.21.15.130:0/2707517226 learned_addr learned my addr 172.21.15.130:0/2707517226 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f991ffff640 1 -- 172.21.15.130:0/2707517226 >> v1:172.21.15.130:6789/0 conn(0x7f992809f290 legacy=0x7f99280a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f991ffff640 1 -- 172.21.15.130:0/2707517226 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992809e7f0 con 0x7f99280a1f90 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.776+0000 7f9925d9b640 1 -- 172.21.15.130:0/2707517226 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2323301174 0 0) 0x7f992809eab0 con 0x7f992809f290 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.777+0000 7f991ffff640 1 --2- 172.21.15.130:0/2707517226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f99280a1f90 0x7f992813a680 crc :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b223e29bdbebcf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.777+0000 7f9925d9b640 1 -- 172.21.15.130:0/2707517226 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9910009d90 con 0x7f99280a1f90 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.777+0000 7f9925d9b640 1 -- 172.21.15.130:0/2707517226 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f991000e230 con 0x7f99280a1f90 2024-04-16T07:49:59.777 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.777+0000 7f9925d9b640 1 -- 172.21.15.130:0/2707517226 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f991000e4d0 con 0x7f99280a1f90 2024-04-16T07:49:59.778 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:49:59.776+0000 7fafa4a34000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.778 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:49:59.777+0000 7fafa4a34000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.779 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.777+0000 7f992cb9d640 1 -- 172.21.15.130:0/2707517226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f99280a1f90 msgr2=0x7f992813a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:49:59.779 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.777+0000 7f992cb9d640 1 --2- 172.21.15.130:0/2707517226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f99280a1f90 0x7f992813a680 crc :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:49:59.779 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.778+0000 7f992cb9d640 1 -- 172.21.15.130:0/2707517226 shutdown_connections 2024-04-16T07:49:59.779 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.778+0000 7f992cb9d640 1 --2- 172.21.15.130:0/2707517226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f99280a1f90 0x7f992813a680 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:49:59.780 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.778+0000 7f992cb9d640 1 -- 172.21.15.130:0/2707517226 >> 172.21.15.130:0/2707517226 conn(0x7f99280a3b20 msgr2=0x7f99280a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:49:59.780 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.778+0000 7f992cb9d640 1 -- 172.21.15.130:0/2707517226 shutdown_connections 2024-04-16T07:49:59.780 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.778+0000 7f992cb9d640 1 -- 172.21.15.130:0/2707517226 wait complete. 2024-04-16T07:49:59.780 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.778+0000 7f992cb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.780 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.780+0000 7f992cb9d640 1 Processor -- start 2024-04-16T07:49:59.780 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.780+0000 7f992cb9d640 1 -- start start 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992cb9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f992809f290 0x7f99281a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992cb9d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f99281a7980 con 0x7f992809f290 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992659c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f992809f290 0x7f99281a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992659c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f992809f290 0x7f99281a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53186/0 (socket says 172.21.15.130:53186) 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992659c640 1 -- 172.21.15.130:0/3888335776 learned_addr learned my addr 172.21.15.130:0/3888335776 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992659c640 1 -- 172.21.15.130:0/3888335776 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99281a7430 con 0x7f992809f290 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992659c640 1 --2- 172.21.15.130:0/3888335776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f992809f290 0x7f99281a4f10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f991effd640 1 -- 172.21.15.130:0/3888335776 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9918004b80 con 0x7f992809f290 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f991effd640 1 -- 172.21.15.130:0/3888335776 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9918004ce0 con 0x7f992809f290 2024-04-16T07:49:59.783 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99281b9170 con 0x7f992809f290 2024-04-16T07:49:59.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f991effd640 1 -- 172.21.15.130:0/3888335776 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9918007120 con 0x7f992809f290 2024-04-16T07:49:59.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.781+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99281a8f20 con 0x7f992809f290 2024-04-16T07:49:59.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.782+0000 7f991effd640 1 -- 172.21.15.130:0/3888335776 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f991800c510 con 0x7f992809f290 2024-04-16T07:49:59.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.782+0000 7f991effd640 1 -- 172.21.15.130:0/3888335776 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (crc 0 0 0) 0x7f991800fa20 con 0x7f992809f290 2024-04-16T07:49:59.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.782+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9928063930 con 0x7f992809f290 2024-04-16T07:49:59.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.785+0000 7f991effd640 1 -- 172.21.15.130:0/3888335776 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7f9918007660 con 0x7f992809f290 2024-04-16T07:49:59.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.848+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f99280739d0 con 0x7f992809f290 2024-04-16T07:49:59.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.849+0000 7f991effd640 1 -- 172.21.15.130:0/3888335776 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (crc 0 0 0) 0x7f991800f230 con 0x7f992809f290 2024-04-16T07:49:59.849 INFO:teuthology.orchestra.run.smithi130.stderr:adjusted tunables profile to default 2024-04-16T07:49:59.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.850+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f992809f290 msgr2=0x7f99281a4f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:49:59.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.850+0000 7f992cb9d640 1 --2- 172.21.15.130:0/3888335776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f992809f290 0x7f99281a4f10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:49:59.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.850+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 shutdown_connections 2024-04-16T07:49:59.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.850+0000 7f992cb9d640 1 --2- 172.21.15.130:0/3888335776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f992809f290 0x7f99281a4f10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:49:59.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.850+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 >> 172.21.15.130:0/3888335776 conn(0x7f99280a3b20 msgr2=0x7f99281bc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:49:59.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.850+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 shutdown_connections 2024-04-16T07:49:59.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.850+0000 7f992cb9d640 1 -- 172.21.15.130:0/3888335776 wait complete. 2024-04-16T07:49:59.863 INFO:tasks.ceph:check_enable_crimson: True 2024-04-16T07:49:59.863 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-04-16T07:49:59.863 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-04-16T07:49:59.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.967+0000 7fa62be64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.973 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.973+0000 7fa62be64640 1 Processor -- start 2024-04-16T07:49:59.973 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.973+0000 7fa62be64640 1 -- start start 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.974+0000 7fa62be64640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa624137ca0 0x7fa62413a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.974+0000 7fa62be64640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6240a1c80 con 0x7fa62409af10 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.974+0000 7fa62be64640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fa6240a10b0 con 0x7fa624137ca0 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.974+0000 7fa6293ff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa624137ca0 0x7fa62413a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.974+0000 7fa629c00640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7fa62409af10 0x7fa6240a22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:37036/0 (socket says 172.21.15.130:37036) 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.974+0000 7fa6293ff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa624137ca0 0x7fa62413a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53188/0 (socket says 172.21.15.130:53188) 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.974+0000 7fa629c00640 1 -- 172.21.15.130:0/1097086739 learned_addr learned my addr 172.21.15.130:0/1097086739 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.975+0000 7fa6293ff640 1 -- 172.21.15.130:0/1097086739 >> v1:172.21.15.130:6789/0 conn(0x7fa62409af10 legacy=0x7fa6240a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:49:59.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.975+0000 7fa6293ff640 1 -- 172.21.15.130:0/1097086739 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240a1970 con 0x7fa624137ca0 2024-04-16T07:49:59.976 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.975+0000 7fa6293ff640 1 --2- 172.21.15.130:0/1097086739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa624137ca0 0x7fa62413a0c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b52939927a2130 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:49:59.976 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.975+0000 7fa628bfe640 1 -- 172.21.15.130:0/1097086739 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa614009d90 con 0x7fa624137ca0 2024-04-16T07:49:59.976 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.975+0000 7fa628bfe640 1 -- 172.21.15.130:0/1097086739 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa61400e230 con 0x7fa624137ca0 2024-04-16T07:49:59.976 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.975+0000 7fa628bfe640 1 -- 172.21.15.130:0/1097086739 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa61400e4d0 con 0x7fa624137ca0 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 1 -- 172.21.15.130:0/1097086739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa624137ca0 msgr2=0x7fa62413a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 1 --2- 172.21.15.130:0/1097086739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa624137ca0 0x7fa62413a0c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 1 -- 172.21.15.130:0/1097086739 shutdown_connections 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 1 --2- 172.21.15.130:0/1097086739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa624137ca0 0x7fa62413a0c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 1 -- 172.21.15.130:0/1097086739 >> 172.21.15.130:0/1097086739 conn(0x7fa6240a3c90 msgr2=0x7fa6240a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 1 -- 172.21.15.130:0/1097086739 shutdown_connections 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 1 -- 172.21.15.130:0/1097086739 wait complete. 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.976+0000 7fa62be64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:49:59.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.977+0000 7fa62be64640 1 Processor -- start 2024-04-16T07:49:59.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.978+0000 7fa62be64640 1 -- start start 2024-04-16T07:49:59.979 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa62be64640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa62409af10 0x7fa62411ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:49:59.979 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa62be64640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa624121570 con 0x7fa62409af10 2024-04-16T07:49:59.979 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa629c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa62409af10 0x7fa62411ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:49:59.979 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa629c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa62409af10 0x7fa62411ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53198/0 (socket says 172.21.15.130:53198) 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa629c00640 1 -- 172.21.15.130:0/686161438 learned_addr learned my addr 172.21.15.130:0/686161438 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa629c00640 1 -- 172.21.15.130:0/686161438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624123020 con 0x7fa62409af10 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa629c00640 1 --2- 172.21.15.130:0/686161438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa62409af10 0x7fa62411ef40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa61a7fc640 1 -- 172.21.15.130:0/686161438 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa620005a60 con 0x7fa62409af10 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa61a7fc640 1 -- 172.21.15.130:0/686161438 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa620005bc0 con 0x7fa62409af10 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa61a7fc640 1 -- 172.21.15.130:0/686161438 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa620005e60 con 0x7fa62409af10 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa624132130 con 0x7fa62409af10 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa624121cf0 con 0x7fa62409af10 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa61a7fc640 1 -- 172.21.15.130:0/686161438 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fa62000cb10 con 0x7fa62409af10 2024-04-16T07:49:59.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.979+0000 7fa61a7fc640 1 -- 172.21.15.130:0/686161438 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (crc 0 0 0) 0x7fa62000f980 con 0x7fa62409af10 2024-04-16T07:49:59.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.980+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa624063930 con 0x7fa62409af10 2024-04-16T07:49:59.982 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:49:59.982+0000 7fa61a7fc640 1 -- 172.21.15.130:0/686161438 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7fa62000fbe0 con 0x7fa62409af10 2024-04-16T07:50:00.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.055+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) -- 0x7fa624132320 con 0x7fa62409af10 2024-04-16T07:50:00.055 INFO:tasks.ceph.mon.a.smithi130.stderr:2024-04-16T07:50:00.056+0000 7f943d7f4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-04-16T07:50:00.055 INFO:tasks.ceph.mon.a.smithi130.stderr:Please be aware that this feature is experimental, untested, 2024-04-16T07:50:00.055 INFO:tasks.ceph.mon.a.smithi130.stderr:unsupported, and may result in data corruption, data loss, 2024-04-16T07:50:00.056 INFO:tasks.ceph.mon.a.smithi130.stderr:and/or irreparable damage to your cluster. Do not use 2024-04-16T07:50:00.056 INFO:tasks.ceph.mon.a.smithi130.stderr:feature with important data. 2024-04-16T07:50:00.056 INFO:tasks.ceph.mon.a.smithi130.stderr: 2024-04-16T07:50:00.245 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:00.246+0000 7fafa4a34000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T07:50:00.298 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:00.298+0000 7fafa4a34000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T07:50:00.813 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:00.813+0000 7fafa4a34000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T07:50:00.850 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.850+0000 7fa61a7fc640 1 -- 172.21.15.130:0/686161438 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) ==== 99+0+0 (crc 0 0 0) 0x7fa62000fd80 con 0x7fa62409af10 2024-04-16T07:50:00.851 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.852+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa62409af10 msgr2=0x7fa62411ef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:00.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.852+0000 7fa62be64640 1 --2- 172.21.15.130:0/686161438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa62409af10 0x7fa62411ef40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:00.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.852+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 shutdown_connections 2024-04-16T07:50:00.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.852+0000 7fa62be64640 1 --2- 172.21.15.130:0/686161438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa62409af10 0x7fa62411ef40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:00.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.852+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 >> 172.21.15.130:0/686161438 conn(0x7fa6240a3c90 msgr2=0x7fa6241347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:00.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.852+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 shutdown_connections 2024-04-16T07:50:00.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:00.852+0000 7fa62be64640 1 -- 172.21.15.130:0/686161438 wait complete. 2024-04-16T07:50:00.865 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-16T07:50:00.865 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:50:00.865 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-16T07:50:00.866 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:00.866+0000 7fafa4a34000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T07:50:00.897 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:50:00.897 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-16T07:50:00.909 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:00.910+0000 7fafa4a34000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T07:50:00.962 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:00.963+0000 7fafa4a34000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T07:50:00.968 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:50:00.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-16T07:50:00.997 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-16T07:50:00.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-16T07:50:01.007 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.008+0000 7fafa4a34000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T07:50:01.035 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a 0 2024-04-16T07:50:01.130 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.131+0000 7fafa4a34000 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T07:50:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.181+0000 7f16db46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.188+0000 7f16db46d640 1 Processor -- start 2024-04-16T07:50:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.188+0000 7f16db46d640 1 -- start start 2024-04-16T07:50:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.188+0000 7f16db46d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f16d410fd80 0x7f16d4110190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.188+0000 7f16db46d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d411d0b0 con 0x7f16d4110790 2024-04-16T07:50:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.188+0000 7f16db46d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f16d411e250 con 0x7f16d410fd80 2024-04-16T07:50:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16d9209640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f16d410fd80 0x7f16d4110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16d8a08640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f16d4110790 0x7f16d411b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.164:45350/0 (socket says 172.21.15.164:45350) 2024-04-16T07:50:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16d9209640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f16d410fd80 0x7f16d4110190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.164:46006/0 (socket says 172.21.15.164:46006) 2024-04-16T07:50:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16d8a08640 1 -- 172.21.15.164:0/2473024295 learned_addr learned my addr 172.21.15.164:0/2473024295 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-04-16T07:50:01.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16d9209640 1 -- 172.21.15.164:0/2473024295 >> v1:172.21.15.130:6789/0 conn(0x7f16d4110790 legacy=0x7f16d411b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16d9209640 1 -- 172.21.15.164:0/2473024295 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d411e390 con 0x7f16d410fd80 2024-04-16T07:50:01.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16d9209640 1 --2- 172.21.15.164:0/2473024295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d4110190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83fe1e673130a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16c7fff640 1 -- 172.21.15.164:0/2473024295 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f16d0009d90 con 0x7f16d410fd80 2024-04-16T07:50:01.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16c7fff640 1 -- 172.21.15.164:0/2473024295 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f16d000e230 con 0x7f16d410fd80 2024-04-16T07:50:01.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.189+0000 7f16c7fff640 1 -- 172.21.15.164:0/2473024295 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f16d000e4d0 con 0x7f16d410fd80 2024-04-16T07:50:01.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.190+0000 7f16db46d640 1 -- 172.21.15.164:0/2473024295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 msgr2=0x7f16d4110190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.190+0000 7f16db46d640 1 --2- 172.21.15.164:0/2473024295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d4110190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.190+0000 7f16db46d640 1 -- 172.21.15.164:0/2473024295 shutdown_connections 2024-04-16T07:50:01.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.190+0000 7f16db46d640 1 --2- 172.21.15.164:0/2473024295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d4110190 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.190+0000 7f16db46d640 1 -- 172.21.15.164:0/2473024295 >> 172.21.15.164:0/2473024295 conn(0x7f16d406fe70 msgr2=0x7f16d4072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.190+0000 7f16db46d640 1 -- 172.21.15.164:0/2473024295 shutdown_connections 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.190+0000 7f16db46d640 1 -- 172.21.15.164:0/2473024295 wait complete. 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.191+0000 7f16db46d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.191+0000 7f16db46d640 1 Processor -- start 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.191+0000 7f16db46d640 1 -- start start 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16db46d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d41ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16db46d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41ac7d0 con 0x7f16d410fd80 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16d9209640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d41ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16d9209640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d41ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.164:46010/0 (socket says 172.21.15.164:46010) 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16d9209640 1 -- 172.21.15.164:0/3620631594 learned_addr learned my addr 172.21.15.164:0/3620631594 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16d9209640 1 -- 172.21.15.164:0/3620631594 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d41ac910 con 0x7f16d410fd80 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16d9209640 1 --2- 172.21.15.164:0/3620631594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d41ac200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16c5ffb640 1 -- 172.21.15.164:0/3620631594 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f16d0014750 con 0x7f16d410fd80 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16c5ffb640 1 -- 172.21.15.164:0/3620631594 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f16d00148b0 con 0x7f16d410fd80 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16c5ffb640 1 -- 172.21.15.164:0/3620631594 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f16d0014b50 con 0x7f16d410fd80 2024-04-16T07:50:01.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.192+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d41acb00 con 0x7f16d410fd80 2024-04-16T07:50:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.193+0000 7f16c5ffb640 1 -- 172.21.15.164:0/3620631594 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f16d0014d50 con 0x7f16d410fd80 2024-04-16T07:50:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.193+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d41a6560 con 0x7f16d410fd80 2024-04-16T07:50:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.193+0000 7f16c5ffb640 1 -- 172.21.15.164:0/3620631594 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 747+0+0 (crc 0 0 0) 0x7f16d0019040 con 0x7f16d410fd80 2024-04-16T07:50:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.194+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d41accf0 con 0x7f16d410fd80 2024-04-16T07:50:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.196+0000 7f16c5ffb640 1 -- 172.21.15.164:0/3620631594 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7f16d0012480 con 0x7f16d410fd80 2024-04-16T07:50:01.205 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.206+0000 7fafa4a34000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T07:50:01.251 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.252+0000 7fafa4a34000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T07:50:01.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.268+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a", "id": 0} v 0) -- 0x7f16d4110790 con 0x7f16d410fd80 2024-04-16T07:50:01.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.269+0000 7f16c5ffb640 1 -- 172.21.15.164:0/3620631594 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a", "id": 0}]=0 v4) ==== 112+0+2 (crc 0 0 0) 0x7f16d0012930 con 0x7f16d410fd80 2024-04-16T07:50:01.269 INFO:teuthology.orchestra.run.smithi164.stdout:0 2024-04-16T07:50:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.270+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 msgr2=0x7f16d41ac200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.270+0000 7f16db46d640 1 --2- 172.21.15.164:0/3620631594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d41ac200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.270+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 shutdown_connections 2024-04-16T07:50:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.270+0000 7f16db46d640 1 --2- 172.21.15.164:0/3620631594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16d410fd80 0x7f16d41ac200 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.270+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 >> 172.21.15.164:0/3620631594 conn(0x7f16d406fe70 msgr2=0x7f16d41142f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.270+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 shutdown_connections 2024-04-16T07:50:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.270+0000 7f16db46d640 1 -- 172.21.15.164:0/3620631594 wait complete. 2024-04-16T07:50:01.281 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 1935374e-a32a-4cda-95d3-af6a8e92f6d3 1 2024-04-16T07:50:01.307 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.307+0000 7fafa4a34000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T07:50:01.357 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.358+0000 7fafa4a34000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T07:50:01.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.387+0000 7fc19b79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.393+0000 7fc19b79d640 1 Processor -- start 2024-04-16T07:50:01.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.393+0000 7fc19b79d640 1 -- start start 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.393+0000 7fc19b79d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fc194113b40 0x7fc194113f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.393+0000 7fc19b79d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc194114710 con 0x7fc194118510 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.393+0000 7fc19b79d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fc194119050 con 0x7fc194113b40 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc199539640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fc194113b40 0x7fc194113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc199539640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fc194113b40 0x7fc194113f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.164:46020/0 (socket says 172.21.15.164:46020) 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc199539640 1 -- 172.21.15.164:0/4028096926 learned_addr learned my addr 172.21.15.164:0/4028096926 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc198d38640 1 --1- 172.21.15.164:0/4028096926 >> v1:172.21.15.130:6789/0 conn(0x7fc194118510 0x7fc194118920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.164:45366/0 (socket says 172.21.15.164:45366) 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc199539640 1 -- 172.21.15.164:0/4028096926 >> v1:172.21.15.130:6789/0 conn(0x7fc194118510 legacy=0x7fc194118920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc199539640 1 -- 172.21.15.164:0/4028096926 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941191c0 con 0x7fc194113b40 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc199539640 1 --2- 172.21.15.164:0/4028096926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc194113f50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c8fa08e20c32dc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc18bfff640 1 -- 172.21.15.164:0/4028096926 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc190009e50 con 0x7fc194113b40 2024-04-16T07:50:01.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc18bfff640 1 -- 172.21.15.164:0/4028096926 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc19000e230 con 0x7fc194113b40 2024-04-16T07:50:01.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.394+0000 7fc18bfff640 1 -- 172.21.15.164:0/4028096926 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc19000e4d0 con 0x7fc194113b40 2024-04-16T07:50:01.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.395+0000 7fc19b79d640 1 -- 172.21.15.164:0/4028096926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 msgr2=0x7fc194113f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.395+0000 7fc19b79d640 1 --2- 172.21.15.164:0/4028096926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc194113f50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.395+0000 7fc19b79d640 1 -- 172.21.15.164:0/4028096926 shutdown_connections 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.395+0000 7fc19b79d640 1 --2- 172.21.15.164:0/4028096926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc194113f50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.395+0000 7fc19b79d640 1 -- 172.21.15.164:0/4028096926 >> 172.21.15.164:0/4028096926 conn(0x7fc19406f100 msgr2=0x7fc194111d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.395+0000 7fc19b79d640 1 -- 172.21.15.164:0/4028096926 shutdown_connections 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.395+0000 7fc19b79d640 1 -- 172.21.15.164:0/4028096926 wait complete. 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.396+0000 7fc19b79d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.396+0000 7fc19b79d640 1 Processor -- start 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.396+0000 7fc19b79d640 1 -- start start 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.397+0000 7fc19b79d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc1941ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.397+0000 7fc19b79d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941ae7c0 con 0x7fc194113b40 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.397+0000 7fc199539640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc1941ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.397+0000 7fc199539640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc1941ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.164:46024/0 (socket says 172.21.15.164:46024) 2024-04-16T07:50:01.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.397+0000 7fc199539640 1 -- 172.21.15.164:0/722063353 learned_addr learned my addr 172.21.15.164:0/722063353 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-16T07:50:01.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.397+0000 7fc199539640 1 -- 172.21.15.164:0/722063353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941ae900 con 0x7fc194113b40 2024-04-16T07:50:01.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.397+0000 7fc199539640 1 --2- 172.21.15.164:0/722063353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc1941ae1f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.398+0000 7fc189ffb640 1 -- 172.21.15.164:0/722063353 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc19000eea0 con 0x7fc194113b40 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.398+0000 7fc189ffb640 1 -- 172.21.15.164:0/722063353 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc190015420 con 0x7fc194113b40 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.398+0000 7fc189ffb640 1 -- 172.21.15.164:0/722063353 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc1900156c0 con 0x7fc194113b40 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.398+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941aeaf0 con 0x7fc194113b40 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.398+0000 7fc189ffb640 1 -- 172.21.15.164:0/722063353 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fc190015c50 con 0x7fc194113b40 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.398+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a85d0 con 0x7fc194113b40 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.398+0000 7fc189ffb640 1 -- 172.21.15.164:0/722063353 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 862+0+0 (crc 0 0 0) 0x7fc190014860 con 0x7fc194113b40 2024-04-16T07:50:01.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.399+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1941aece0 con 0x7fc194113b40 2024-04-16T07:50:01.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.401+0000 7fc189ffb640 1 -- 172.21.15.164:0/722063353 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7fc190014a20 con 0x7fc194113b40 2024-04-16T07:50:01.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.472+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1935374e-a32a-4cda-95d3-af6a8e92f6d3", "id": 1} v 0) -- 0x7fc1941185b0 con 0x7fc194113b40 2024-04-16T07:50:01.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.474+0000 7fc189ffb640 1 -- 172.21.15.164:0/722063353 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1935374e-a32a-4cda-95d3-af6a8e92f6d3", "id": 1}]=0 v5) ==== 112+0+2 (crc 0 0 0) 0x7fc190014c40 con 0x7fc194113b40 2024-04-16T07:50:01.473 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-04-16T07:50:01.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.475+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 msgr2=0x7fc1941ae1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.475+0000 7fc19b79d640 1 --2- 172.21.15.164:0/722063353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc1941ae1f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.475+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 shutdown_connections 2024-04-16T07:50:01.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.475+0000 7fc19b79d640 1 --2- 172.21.15.164:0/722063353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc194113b40 0x7fc1941ae1f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.475+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 >> 172.21.15.164:0/722063353 conn(0x7fc19406f100 msgr2=0x7fc194111f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.475+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 shutdown_connections 2024-04-16T07:50:01.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.475+0000 7fc19b79d640 1 -- 172.21.15.164:0/722063353 wait complete. 2024-04-16T07:50:01.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 0e51d8b6-100a-42fa-b7cf-9d57d230ee1f 2 2024-04-16T07:50:01.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.632+0000 7fa2a722d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.638+0000 7fa2a722d640 1 Processor -- start 2024-04-16T07:50:01.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.638+0000 7fa2a722d640 1 -- start start 2024-04-16T07:50:01.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.638+0000 7fa2a722d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa2a011c780 0x7fa2a011eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.638+0000 7fa2a722d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a011f170 con 0x7fa2a0118800 2024-04-16T07:50:01.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.638+0000 7fa2a722d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a0120330 con 0x7fa2a011c780 2024-04-16T07:50:01.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.639+0000 7fa29ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa2a011c780 0x7fa2a011eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.639+0000 7fa2a4fc9640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7fa2a0118800 0x7fa2a0118c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.164:45368/0 (socket says 172.21.15.164:45368) 2024-04-16T07:50:01.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.639+0000 7fa2a4fc9640 1 -- 172.21.15.164:0/3378716112 learned_addr learned my addr 172.21.15.164:0/3378716112 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-04-16T07:50:01.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.639+0000 7fa29ffff640 1 -- 172.21.15.164:0/3378716112 >> v1:172.21.15.130:6789/0 conn(0x7fa2a0118800 legacy=0x7fa2a0118c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.639+0000 7fa29ffff640 1 -- 172.21.15.164:0/3378716112 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01204a0 con 0x7fa2a011c780 2024-04-16T07:50:01.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.639+0000 7fa29ffff640 1 --2- 172.21.15.164:0/3378716112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a011c780 0x7fa2a011eba0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8ef8fc6ac0a137 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa29f7fe640 1 -- 172.21.15.164:0/3378716112 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa28c009d90 con 0x7fa2a011c780 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa29f7fe640 1 -- 172.21.15.164:0/3378716112 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa28c00e230 con 0x7fa2a011c780 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa29f7fe640 1 -- 172.21.15.164:0/3378716112 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa28c00e4d0 con 0x7fa2a011c780 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa2a722d640 1 -- 172.21.15.164:0/3378716112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a011c780 msgr2=0x7fa2a011eba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa2a722d640 1 --2- 172.21.15.164:0/3378716112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a011c780 0x7fa2a011eba0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa2a722d640 1 -- 172.21.15.164:0/3378716112 shutdown_connections 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa2a722d640 1 --2- 172.21.15.164:0/3378716112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a011c780 0x7fa2a011eba0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa2a722d640 1 -- 172.21.15.164:0/3378716112 >> 172.21.15.164:0/3378716112 conn(0x7fa2a006f0e0 msgr2=0x7fa2a0111d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa2a722d640 1 -- 172.21.15.164:0/3378716112 shutdown_connections 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.640+0000 7fa2a722d640 1 -- 172.21.15.164:0/3378716112 wait complete. 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.641+0000 7fa2a722d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.641+0000 7fa2a722d640 1 Processor -- start 2024-04-16T07:50:01.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.641+0000 7fa2a722d640 1 -- start start 2024-04-16T07:50:01.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.642+0000 7fa2a722d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a0118800 0x7fa2a01ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.642+0000 7fa2a722d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ae780 con 0x7fa2a0118800 2024-04-16T07:50:01.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.642+0000 7fa2a4fc9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a0118800 0x7fa2a01ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.642+0000 7fa2a4fc9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a0118800 0x7fa2a01ae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.164:46034/0 (socket says 172.21.15.164:46034) 2024-04-16T07:50:01.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.642+0000 7fa2a4fc9640 1 -- 172.21.15.164:0/314593494 learned_addr learned my addr 172.21.15.164:0/314593494 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-16T07:50:01.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.642+0000 7fa2a4fc9640 1 -- 172.21.15.164:0/314593494 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01ae8c0 con 0x7fa2a0118800 2024-04-16T07:50:01.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa2a4fc9640 1 --2- 172.21.15.164:0/314593494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a0118800 0x7fa2a01ae1b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa29d7fa640 1 -- 172.21.15.164:0/314593494 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa294005a60 con 0x7fa2a0118800 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa29d7fa640 1 -- 172.21.15.164:0/314593494 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa294005bc0 con 0x7fa2a0118800 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa29d7fa640 1 -- 172.21.15.164:0/314593494 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa294005e60 con 0x7fa2a0118800 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a01aeb10 con 0x7fa2a0118800 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a8510 con 0x7fa2a0118800 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa29d7fa640 1 -- 172.21.15.164:0/314593494 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fa29400cb80 con 0x7fa2a0118800 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.643+0000 7fa29d7fa640 1 -- 172.21.15.164:0/314593494 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 977+0+0 (crc 0 0 0) 0x7fa29400fbc0 con 0x7fa2a0118800 2024-04-16T07:50:01.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.644+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a0063930 con 0x7fa2a0118800 2024-04-16T07:50:01.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.646+0000 7fa29d7fa640 1 -- 172.21.15.164:0/314593494 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7fa29400f230 con 0x7fa2a0118800 2024-04-16T07:50:01.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.717+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0e51d8b6-100a-42fa-b7cf-9d57d230ee1f", "id": 2} v 0) -- 0x7fa2a011c850 con 0x7fa2a0118800 2024-04-16T07:50:01.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.719+0000 7fa29d7fa640 1 -- 172.21.15.164:0/314593494 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e51d8b6-100a-42fa-b7cf-9d57d230ee1f", "id": 2}]=0 v6) ==== 112+0+2 (crc 0 0 0) 0x7fa29400f780 con 0x7fa2a0118800 2024-04-16T07:50:01.718 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-04-16T07:50:01.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.720+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a0118800 msgr2=0x7fa2a01ae1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.720+0000 7fa2a722d640 1 --2- 172.21.15.164:0/314593494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a0118800 0x7fa2a01ae1b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.720+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 shutdown_connections 2024-04-16T07:50:01.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.720+0000 7fa2a722d640 1 --2- 172.21.15.164:0/314593494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2a0118800 0x7fa2a01ae1b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.720+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 >> 172.21.15.164:0/314593494 conn(0x7fa2a006f0e0 msgr2=0x7fa2a011a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.720+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 shutdown_connections 2024-04-16T07:50:01.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.720+0000 7fa2a722d640 1 -- 172.21.15.164:0/314593494 wait complete. 2024-04-16T07:50:01.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 9bc937c4-1eec-4856-965b-3de26632e115 3 2024-04-16T07:50:01.752 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.753+0000 7fafa4a34000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T07:50:01.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.837+0000 7fa8c839d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.843+0000 7fa8c839d640 1 Processor -- start 2024-04-16T07:50:01.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.843+0000 7fa8c839d640 1 -- start start 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c839d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa8c011c780 0x7fa8c011eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c839d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c011f170 con 0x7fa8c0118800 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c839d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c0120330 con 0x7fa8c011c780 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c5938640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa8c011c780 0x7fa8c011eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c5938640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa8c011c780 0x7fa8c011eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.164:46046/0 (socket says 172.21.15.164:46046) 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c5938640 1 -- 172.21.15.164:0/685093177 learned_addr learned my addr 172.21.15.164:0/685093177 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c6139640 1 --1- 172.21.15.164:0/685093177 >> v1:172.21.15.130:6789/0 conn(0x7fa8c0118800 0x7fa8c0118c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.164:45376/0 (socket says 172.21.15.164:45376) 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c5938640 1 -- 172.21.15.164:0/685093177 >> v1:172.21.15.130:6789/0 conn(0x7fa8c0118800 legacy=0x7fa8c0118c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c5938640 1 -- 172.21.15.164:0/685093177 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c01204a0 con 0x7fa8c011c780 2024-04-16T07:50:01.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.844+0000 7fa8c5938640 1 --2- 172.21.15.164:0/685093177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c011c780 0x7fa8c011eba0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e2d3b090210ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.845+0000 7fa8c5137640 1 -- 172.21.15.164:0/685093177 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa8b0009e50 con 0x7fa8c011c780 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.845+0000 7fa8c5137640 1 -- 172.21.15.164:0/685093177 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa8b000e230 con 0x7fa8c011c780 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.845+0000 7fa8c5137640 1 -- 172.21.15.164:0/685093177 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa8b000e4d0 con 0x7fa8c011c780 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.845+0000 7fa8c839d640 1 -- 172.21.15.164:0/685093177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c011c780 msgr2=0x7fa8c011eba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.845+0000 7fa8c839d640 1 --2- 172.21.15.164:0/685093177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c011c780 0x7fa8c011eba0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 1 -- 172.21.15.164:0/685093177 shutdown_connections 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 1 --2- 172.21.15.164:0/685093177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c011c780 0x7fa8c011eba0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 1 -- 172.21.15.164:0/685093177 >> 172.21.15.164:0/685093177 conn(0x7fa8c006f0e0 msgr2=0x7fa8c0111d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 1 -- 172.21.15.164:0/685093177 shutdown_connections 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 1 -- 172.21.15.164:0/685093177 wait complete. 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 1 Processor -- start 2024-04-16T07:50:01.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.846+0000 7fa8c839d640 1 -- start start 2024-04-16T07:50:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.847+0000 7fa8c839d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c0118800 0x7fa8c01ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.847+0000 7fa8c839d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01ae780 con 0x7fa8c0118800 2024-04-16T07:50:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.847+0000 7fa8c6139640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c0118800 0x7fa8c01ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.847+0000 7fa8c6139640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c0118800 0x7fa8c01ae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.164:46050/0 (socket says 172.21.15.164:46050) 2024-04-16T07:50:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.847+0000 7fa8c6139640 1 -- 172.21.15.164:0/3068851597 learned_addr learned my addr 172.21.15.164:0/3068851597 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-16T07:50:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.847+0000 7fa8c6139640 1 -- 172.21.15.164:0/3068851597 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c01ae8c0 con 0x7fa8c0118800 2024-04-16T07:50:01.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8c6139640 1 --2- 172.21.15.164:0/3068851597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c0118800 0x7fa8c01ae1b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8b6ffd640 1 -- 172.21.15.164:0/3068851597 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa8bc0059a0 con 0x7fa8c0118800 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8b6ffd640 1 -- 172.21.15.164:0/3068851597 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa8bc005b00 con 0x7fa8c0118800 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c01aeb10 con 0x7fa8c0118800 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8b6ffd640 1 -- 172.21.15.164:0/3068851597 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa8bc005da0 con 0x7fa8c0118800 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c01a8510 con 0x7fa8c0118800 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8b6ffd640 1 -- 172.21.15.164:0/3068851597 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fa8bc00cc00 con 0x7fa8c0118800 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.848+0000 7fa8b6ffd640 1 -- 172.21.15.164:0/3068851597 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1092+0+0 (crc 0 0 0) 0x7fa8bc00f230 con 0x7fa8c0118800 2024-04-16T07:50:01.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.849+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c01aeca0 con 0x7fa8c0118800 2024-04-16T07:50:01.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.851+0000 7fa8b6ffd640 1 -- 172.21.15.164:0/3068851597 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7fa8bc0070b0 con 0x7fa8c0118800 2024-04-16T07:50:01.887 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:01.888+0000 7fafa4a34000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T07:50:01.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.923+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9bc937c4-1eec-4856-965b-3de26632e115", "id": 3} v 0) -- 0x7fa8c011c7b0 con 0x7fa8c0118800 2024-04-16T07:50:01.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.927+0000 7fa8b6ffd640 1 -- 172.21.15.164:0/3068851597 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9bc937c4-1eec-4856-965b-3de26632e115", "id": 3}]=0 v7) ==== 112+0+2 (crc 0 0 0) 0x7fa8bc0072d0 con 0x7fa8c0118800 2024-04-16T07:50:01.927 INFO:teuthology.orchestra.run.smithi164.stdout:3 2024-04-16T07:50:01.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.928+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c0118800 msgr2=0x7fa8c01ae1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:01.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.928+0000 7fa8c839d640 1 --2- 172.21.15.164:0/3068851597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c0118800 0x7fa8c01ae1b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.928+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 shutdown_connections 2024-04-16T07:50:01.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.928+0000 7fa8c839d640 1 --2- 172.21.15.164:0/3068851597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa8c0118800 0x7fa8c01ae1b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:01.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.928+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 >> 172.21.15.164:0/3068851597 conn(0x7fa8c006f0e0 msgr2=0x7fa8c011a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:01.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.928+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 shutdown_connections 2024-04-16T07:50:01.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-16T07:50:01.928+0000 7fa8c839d640 1 -- 172.21.15.164:0/3068851597 wait complete. 2024-04-16T07:50:01.939 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-04-16T07:50:01.977 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-16T07:50:01.977 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-04-16T07:50:02.005 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:02.006+0000 7fafa4a34000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T07:50:02.020 INFO:tasks.ceph.osd.0:Started 2024-04-16T07:50:02.020 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-04-16T07:50:02.050 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:02.051+0000 7fafa4a34000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T07:50:02.055 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-16T07:50:02.056 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-04-16T07:50:02.098 INFO:tasks.ceph.osd.1:Started 2024-04-16T07:50:02.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-04-16T07:50:02.132 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-16T07:50:02.132 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-04-16T07:50:02.165 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:02.165+0000 7fafa4a34000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T07:50:02.166 INFO:tasks.ceph.osd.2:Started 2024-04-16T07:50:02.167 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-04-16T07:50:02.199 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-16T07:50:02.199 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-04-16T07:50:02.241 INFO:tasks.ceph.osd.3:Started 2024-04-16T07:50:02.242 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T07:50:02.268 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:02.269+0000 7fafa4a34000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T07:50:02.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.359+0000 7fe7fd39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:02.363 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.364+0000 7fe7fd39d640 1 Processor -- start 2024-04-16T07:50:02.363 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.364+0000 7fe7fd39d640 1 -- start start 2024-04-16T07:50:02.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7fd39d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fe7f8131c20 0x7fe7f8132030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:02.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7fd39d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f81319a0 con 0x7fe7f81349e0 2024-04-16T07:50:02.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7fd39d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80a2b70 con 0x7fe7f8131c20 2024-04-16T07:50:02.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7f6ffd640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fe7f8131c20 0x7fe7f8132030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7ee7fc640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7fe7f81349e0 0x7fe7f8134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:37046/0 (socket says 172.21.15.130:37046) 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7ee7fc640 1 -- 172.21.15.130:0/270851702 learned_addr learned my addr 172.21.15.130:0/270851702 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7f6ffd640 1 -- 172.21.15.130:0/270851702 >> v1:172.21.15.130:6789/0 conn(0x7fe7f81349e0 legacy=0x7fe7f8134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.365+0000 7fe7f6ffd640 1 -- 172.21.15.130:0/270851702 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f809e570 con 0x7fe7f8131c20 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.366+0000 7fe7f6ffd640 1 --2- 172.21.15.130:0/270851702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f8132030 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca0f8c585d64eca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.366+0000 7fe7f67fc640 1 -- 172.21.15.130:0/270851702 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe7e8009d90 con 0x7fe7f8131c20 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.366+0000 7fe7f67fc640 1 -- 172.21.15.130:0/270851702 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe7e800e230 con 0x7fe7f8131c20 2024-04-16T07:50:02.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.366+0000 7fe7f67fc640 1 -- 172.21.15.130:0/270851702 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe7e800e4d0 con 0x7fe7f8131c20 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 1 -- 172.21.15.130:0/270851702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 msgr2=0x7fe7f8132030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 1 --2- 172.21.15.130:0/270851702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f8132030 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 1 -- 172.21.15.130:0/270851702 shutdown_connections 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 1 --2- 172.21.15.130:0/270851702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f8132030 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 1 -- 172.21.15.130:0/270851702 >> 172.21.15.130:0/270851702 conn(0x7fe7f80a3d50 msgr2=0x7fe7f80a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 1 -- 172.21.15.130:0/270851702 shutdown_connections 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 1 -- 172.21.15.130:0/270851702 wait complete. 2024-04-16T07:50:02.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.367+0000 7fe7fd39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:02.368 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.368+0000 7fe7fd39d640 1 Processor -- start 2024-04-16T07:50:02.368 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.368+0000 7fe7fd39d640 1 -- start start 2024-04-16T07:50:02.370 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.369+0000 7fe7fd39d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f811f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:02.370 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.369+0000 7fe7fd39d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f8121610 con 0x7fe7f8131c20 2024-04-16T07:50:02.370 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7f6ffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f811f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7f6ffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f811f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53218/0 (socket says 172.21.15.130:53218) 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7f6ffd640 1 -- 172.21.15.130:0/1774510172 learned_addr learned my addr 172.21.15.130:0/1774510172 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7f6ffd640 1 -- 172.21.15.130:0/1774510172 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f8122b50 con 0x7fe7f8131c20 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7f6ffd640 1 --2- 172.21.15.130:0/1774510172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f811f660 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7effff640 1 -- 172.21.15.130:0/1774510172 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe7e8014750 con 0x7fe7f8131c20 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7effff640 1 -- 172.21.15.130:0/1774510172 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe7e80148b0 con 0x7fe7f8131c20 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f8122840 con 0x7fe7f8131c20 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7effff640 1 -- 172.21.15.130:0/1774510172 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe7e8014b50 con 0x7fe7f8131c20 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f8123270 con 0x7fe7f8131c20 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7effff640 1 -- 172.21.15.130:0/1774510172 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fe7e8014d50 con 0x7fe7f8131c20 2024-04-16T07:50:02.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.370+0000 7fe7effff640 1 -- 172.21.15.130:0/1774510172 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7fe7e8019040 con 0x7fe7f8131c20 2024-04-16T07:50:02.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.371+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f8063930 con 0x7fe7f8131c20 2024-04-16T07:50:02.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.373+0000 7fe7effff640 1 -- 172.21.15.130:0/1774510172 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7fe7e8013970 con 0x7fe7f8131c20 2024-04-16T07:50:02.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.435+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7f809dec0 con 0x7fe7f8131c20 2024-04-16T07:50:02.435 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.435+0000 7fe7effff640 1 -- 172.21.15.130:0/1774510172 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7fe7e8013b10 con 0x7fe7f8131c20 2024-04-16T07:50:02.435 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:50:02.435 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":7,"fsid":"06ce95cb-daea-4a00-b123-d08aef1a06ab","created":"2024-04-16T07:49:59.765016+0000","modified":"2024-04-16T07:50:01.924952+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T07:50:01.924952+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1935374e-a32a-4cda-95d3-af6a8e92f6d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0e51d8b6-100a-42fa-b7cf-9d57d230ee1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9bc937c4-1eec-4856-965b-3de26632e115","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T07:50:02.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.436+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 msgr2=0x7fe7f811f660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:02.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.436+0000 7fe7fd39d640 1 --2- 172.21.15.130:0/1774510172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f811f660 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:02.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.436+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 shutdown_connections 2024-04-16T07:50:02.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.436+0000 7fe7fd39d640 1 --2- 172.21.15.130:0/1774510172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f8131c20 0x7fe7f811f660 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:02.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.436+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 >> 172.21.15.130:0/1774510172 conn(0x7fe7f80a3d50 msgr2=0x7fe7f8130d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:02.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.436+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 shutdown_connections 2024-04-16T07:50:02.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:02.436+0000 7fe7fd39d640 1 -- 172.21.15.130:0/1774510172 wait complete. 2024-04-16T07:50:02.445 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:02.445+0000 7fafa4a34000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T07:50:02.447 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-16T07:50:02.447 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-16T07:50:02.492 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:02.493+0000 7fafa4a34000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T07:50:02.769 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:02.769+0000 7fafa4a34000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T07:50:02.849 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-16T07:50:02.956 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.952+0000 7f5f2f75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:02.957 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.957+0000 7f5f2f75d640 1 Processor -- start 2024-04-16T07:50:02.957 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.958+0000 7f5f2f75d640 1 -- start start 2024-04-16T07:50:02.959 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2f75d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f5f2809f7d0 0x7f5f2809fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:02.959 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2f75d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f2809ee40 con 0x7f5f28134ad0 2024-04-16T07:50:02.959 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2f75d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f5f2809f380 con 0x7f5f2809f7d0 2024-04-16T07:50:02.959 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2d4f9640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f5f2809f7d0 0x7f5f2809fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:02.959 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2ccf8640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f5f28134ad0 0x7f5f28134ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:37050/0 (socket says 172.21.15.130:37050) 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2ccf8640 1 -- 172.21.15.130:0/2301174776 learned_addr learned my addr 172.21.15.130:0/2301174776 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2d4f9640 1 -- 172.21.15.130:0/2301174776 >> v1:172.21.15.130:6789/0 conn(0x7f5f28134ad0 legacy=0x7f5f28134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2d4f9640 1 -- 172.21.15.130:0/2301174776 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2809f070 con 0x7f5f2809f7d0 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f1bfff640 1 -- 172.21.15.130:0/2301174776 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3211257707 0 0) 0x7f5f2809ee40 con 0x7f5f28134ad0 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f2d4f9640 1 --2- 172.21.15.130:0/2301174776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f2809fbe0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd9c463c472a54a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.959+0000 7f5f1bfff640 1 -- 172.21.15.130:0/2301174776 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5f24009d90 con 0x7f5f2809f7d0 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.960+0000 7f5f1bfff640 1 -- 172.21.15.130:0/2301174776 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5f2400e230 con 0x7f5f2809f7d0 2024-04-16T07:50:02.960 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.960+0000 7f5f1bfff640 1 -- 172.21.15.130:0/2301174776 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5f2400e4d0 con 0x7f5f2809f7d0 2024-04-16T07:50:02.961 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.960+0000 7f5f2f75d640 1 -- 172.21.15.130:0/2301174776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 msgr2=0x7f5f2809fbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:02.961 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.960+0000 7f5f2f75d640 1 --2- 172.21.15.130:0/2301174776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f2809fbe0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:02.961 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.960+0000 7f5f2f75d640 1 -- 172.21.15.130:0/2301174776 shutdown_connections 2024-04-16T07:50:02.961 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.960+0000 7f5f2f75d640 1 --2- 172.21.15.130:0/2301174776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f2809fbe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:02.961 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.961+0000 7f5f2f75d640 1 -- 172.21.15.130:0/2301174776 >> 172.21.15.130:0/2301174776 conn(0x7f5f280a5130 msgr2=0x7f5f280a5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:02.961 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.961+0000 7f5f2f75d640 1 -- 172.21.15.130:0/2301174776 shutdown_connections 2024-04-16T07:50:02.962 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.961+0000 7f5f2f75d640 1 -- 172.21.15.130:0/2301174776 wait complete. 2024-04-16T07:50:02.962 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.961+0000 7f5f2f75d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:02.962 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.962+0000 7f5f2f75d640 1 Processor -- start 2024-04-16T07:50:02.962 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.962+0000 7f5f2f75d640 1 -- start start 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.963+0000 7f5f2f75d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f281204b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.963+0000 7f5f2f75d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28121550 con 0x7f5f2809f7d0 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.963+0000 7f5f2d4f9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f281204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.963+0000 7f5f2d4f9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f281204b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53240/0 (socket says 172.21.15.130:53240) 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.963+0000 7f5f2d4f9640 1 -- 172.21.15.130:0/1005213282 learned_addr learned my addr 172.21.15.130:0/1005213282 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f2d4f9640 1 -- 172.21.15.130:0/1005213282 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f28123f70 con 0x7f5f2809f7d0 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f2d4f9640 1 --2- 172.21.15.130:0/1005213282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f281204b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f19ffb640 1 -- 172.21.15.130:0/1005213282 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5f24014620 con 0x7f5f2809f7d0 2024-04-16T07:50:02.964 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f19ffb640 1 -- 172.21.15.130:0/1005213282 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5f24014780 con 0x7f5f2809f7d0 2024-04-16T07:50:02.965 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f281241a0 con 0x7f5f2809f7d0 2024-04-16T07:50:02.965 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f19ffb640 1 -- 172.21.15.130:0/1005213282 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5f24014a20 con 0x7f5f2809f7d0 2024-04-16T07:50:02.965 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281230e0 con 0x7f5f2809f7d0 2024-04-16T07:50:02.965 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f19ffb640 1 -- 172.21.15.130:0/1005213282 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f5f24013400 con 0x7f5f2809f7d0 2024-04-16T07:50:02.965 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.964+0000 7f5f19ffb640 1 -- 172.21.15.130:0/1005213282 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7f5f24019040 con 0x7f5f2809f7d0 2024-04-16T07:50:02.965 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.966+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f28124390 con 0x7f5f2809f7d0 2024-04-16T07:50:02.967 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:02.968+0000 7f5f19ffb640 1 -- 172.21.15.130:0/1005213282 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (crc 0 0 0) 0x7f5f24011380 con 0x7f5f2809f7d0 2024-04-16T07:50:03.029 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.030+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f2806f940 con 0x7f5f2809f7d0 2024-04-16T07:50:03.030 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.030+0000 7f5f19ffb640 1 -- 172.21.15.130:0/1005213282 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7f5f24011920 con 0x7f5f2809f7d0 2024-04-16T07:50:03.030 INFO:teuthology.misc.health.smithi130.stdout: 2024-04-16T07:50:03.030 INFO:teuthology.misc.health.smithi130.stdout:{"epoch":7,"fsid":"06ce95cb-daea-4a00-b123-d08aef1a06ab","created":"2024-04-16T07:49:59.765016+0000","modified":"2024-04-16T07:50:01.924952+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T07:50:01.924952+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1935374e-a32a-4cda-95d3-af6a8e92f6d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0e51d8b6-100a-42fa-b7cf-9d57d230ee1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9bc937c4-1eec-4856-965b-3de26632e115","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T07:50:03.031 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.031+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 msgr2=0x7f5f281204b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:03.031 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.031+0000 7f5f2f75d640 1 --2- 172.21.15.130:0/1005213282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f281204b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:03.031 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.031+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 shutdown_connections 2024-04-16T07:50:03.031 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.031+0000 7f5f2f75d640 1 --2- 172.21.15.130:0/1005213282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f2809f7d0 0x7f5f281204b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:03.031 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.031+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 >> 172.21.15.130:0/1005213282 conn(0x7f5f280a5130 msgr2=0x7f5f281bcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:03.032 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.031+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 shutdown_connections 2024-04-16T07:50:03.032 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:03.031+0000 7f5f2f75d640 1 -- 172.21.15.130:0/1005213282 wait complete. 2024-04-16T07:50:03.041 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-16T07:50:03.679 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:03.679+0000 7fafa4a34000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T07:50:03.965 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:03.966+0000 7fafa4a34000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T07:50:04.019 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:04.020+0000 7fafa4a34000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T07:50:04.154 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:04.155+0000 7fafa4a34000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T07:50:05.169 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:50:05.170+0000 7faf43876640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-16T07:50:09.445 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-16T07:50:09.553 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.549+0000 7fe0f1e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:09.554 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.554+0000 7fe0f1e6d640 1 Processor -- start 2024-04-16T07:50:09.554 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.554+0000 7fe0f1e6d640 1 -- start start 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.555+0000 7fe0f1e6d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fe0ec09f7d0 0x7fe0ec09fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.555+0000 7fe0f1e6d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ec09ee40 con 0x7fe0ec134ad0 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.555+0000 7fe0f1e6d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ec09f380 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.555+0000 7fe0eb7fe640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fe0ec09f7d0 0x7fe0ec09fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0e3fff640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7fe0ec134ad0 0x7fe0ec134ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42454/0 (socket says 172.21.15.130:42454) 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0e3fff640 1 -- 172.21.15.130:0/4024552186 learned_addr learned my addr 172.21.15.130:0/4024552186 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0eb7fe640 1 -- 172.21.15.130:0/4024552186 >> v1:172.21.15.130:6789/0 conn(0x7fe0ec134ad0 legacy=0x7fe0ec134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:09.556 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0eb7fe640 1 -- 172.21.15.130:0/4024552186 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec09f070 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.557 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0eb7fe640 1 --2- 172.21.15.130:0/4024552186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec09fbe0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276157da321b2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:09.557 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0eaffd640 1 -- 172.21.15.130:0/4024552186 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0dc009d90 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.557 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0eaffd640 1 -- 172.21.15.130:0/4024552186 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe0dc00e230 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.557 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.556+0000 7fe0eaffd640 1 -- 172.21.15.130:0/4024552186 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0dc00e4d0 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.558 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/4024552186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 msgr2=0x7fe0ec09fbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:09.558 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 1 --2- 172.21.15.130:0/4024552186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec09fbe0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.558 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/4024552186 shutdown_connections 2024-04-16T07:50:09.558 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 1 --2- 172.21.15.130:0/4024552186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec09fbe0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.558 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/4024552186 >> 172.21.15.130:0/4024552186 conn(0x7fe0ec0a5130 msgr2=0x7fe0ec0a5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:09.558 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/4024552186 shutdown_connections 2024-04-16T07:50:09.559 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/4024552186 wait complete. 2024-04-16T07:50:09.559 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.557+0000 7fe0f1e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:09.559 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.559+0000 7fe0f1e6d640 1 Processor -- start 2024-04-16T07:50:09.559 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.559+0000 7fe0f1e6d640 1 -- start start 2024-04-16T07:50:09.560 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0f1e6d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:09.560 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0f1e6d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec122dd0 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.560 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:09.560 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec120620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57980/0 (socket says 172.21.15.130:57980) 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0eb7fe640 1 -- 172.21.15.130:0/3992548909 learned_addr learned my addr 172.21.15.130:0/3992548909 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0eb7fe640 1 -- 172.21.15.130:0/3992548909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec134600 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0eb7fe640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec120620 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0e8ff9640 1 -- 172.21.15.130:0/3992548909 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0dc014620 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0e8ff9640 1 -- 172.21.15.130:0/3992548909 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe0dc014780 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0e8ff9640 1 -- 172.21.15.130:0/3992548909 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0dc014a20 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec124630 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.561 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.560+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec123220 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.562 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.561+0000 7fe0e8ff9640 1 -- 172.21.15.130:0/3992548909 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 4) ==== 45778+0+0 (crc 0 0 0) 0x7fe0dc014bd0 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.562 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.561+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ec063930 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.562 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.562+0000 7fe0e8ff9640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe0cc0388f0 0x7fe0cc03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:09.562 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.562+0000 7fe0e8ff9640 1 -- 172.21.15.130:0/3992548909 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2319+0+0 (crc 0 0 0) 0x7fe0dc019040 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.562 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.562+0000 7fe0e3fff640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe0cc0388f0 0x7fe0cc03ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:09.562 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.562+0000 7fe0e3fff640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe0cc0388f0 0x7fe0cc03ae00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:09.567 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.568+0000 7fe0e8ff9640 1 -- 172.21.15.130:0/3992548909 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7fe0dc018190 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.750 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.750+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0ec134790 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.751 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.751+0000 7fe0e8ff9640 1 -- 172.21.15.130:0/3992548909 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+6963 (crc 0 0 0) 0x7fe0dc00fb30 con 0x7fe0ec09f7d0 2024-04-16T07:50:09.751 INFO:teuthology.misc.health.smithi130.stdout: 2024-04-16T07:50:09.751 INFO:teuthology.misc.health.smithi130.stdout:{"epoch":11,"fsid":"06ce95cb-daea-4a00-b123-d08aef1a06ab","created":"2024-04-16T07:49:59.765016+0000","modified":"2024-04-16T07:50:09.213129+0000","last_up_change":"2024-04-16T07:50:06.203593+0000","last_in_change":"2024-04-16T07:50:01.924952+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T07:50:08.183717+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3361768573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6801","nonce":3361768573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":3361768573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6809","nonce":3361768573}]},"public_addr":"172.21.15.130:6802/3361768573","cluster_addr":"172.21.15.130:6801/3361768573","heartbeat_back_addr":"172.21.15.130:6808/3361768573","heartbeat_front_addr":"172.21.15.130:6809/3361768573","state":["exists","up"]},{"osd":1,"uuid":"1935374e-a32a-4cda-95d3-af6a8e92f6d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3830191025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6805","nonce":3830191025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3830191025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6803","nonce":3830191025}]},"public_addr":"172.21.15.130:6804/3830191025","cluster_addr":"172.21.15.130:6805/3830191025","heartbeat_back_addr":"172.21.15.130:6806/3830191025","heartbeat_front_addr":"172.21.15.130:6803/3830191025","state":["exists","up"]},{"osd":2,"uuid":"0e51d8b6-100a-42fa-b7cf-9d57d230ee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2835634474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6805","nonce":2835634474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":2835634474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2835634474}]},"public_addr":"172.21.15.164:6804/2835634474","cluster_addr":"172.21.15.164:6805/2835634474","heartbeat_back_addr":"172.21.15.164:6815/2835634474","heartbeat_front_addr":"172.21.15.164:6816/2835634474","state":["exists","up"]},{"osd":3,"uuid":"9bc937c4-1eec-4856-965b-3de26632e115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3962925382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6803","nonce":3962925382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6807","nonce":3962925382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3962925382}]},"public_addr":"172.21.15.164:6806/3962925382","cluster_addr":"172.21.15.164:6803/3962925382","heartbeat_back_addr":"172.21.15.164:6807/3962925382","heartbeat_front_addr":"172.21.15.164:6802/3962925382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T07:50:09.753 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe0cc0388f0 msgr2=0x7fe0cc03ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:09.753 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe0cc0388f0 0x7fe0cc03ae00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.753 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 msgr2=0x7fe0ec120620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:09.753 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec120620 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.753 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 shutdown_connections 2024-04-16T07:50:09.753 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe0cc0388f0 0x7fe0cc03ae00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.753 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 --2- 172.21.15.130:0/3992548909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0ec09f7d0 0x7fe0ec120620 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.754 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 >> 172.21.15.130:0/3992548909 conn(0x7fe0ec0a5130 msgr2=0x7fe0ec1bcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:09.754 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 shutdown_connections 2024-04-16T07:50:09.754 INFO:teuthology.misc.health.smithi130.stderr:2024-04-16T07:50:09.753+0000 7fe0f1e6d640 1 -- 172.21.15.130:0/3992548909 wait complete. 2024-04-16T07:50:09.763 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-16T07:50:09.763 INFO:tasks.ceph:Creating RBD pool 2024-04-16T07:50:09.763 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-16T07:50:09.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.874+0000 7f343139d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:09.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.883+0000 7f343139d640 1 Processor -- start 2024-04-16T07:50:09.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.883+0000 7f343139d640 1 -- start start 2024-04-16T07:50:09.884 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.884+0000 7f343139d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f342c0a1f90 0x7f342c13a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.884+0000 7f343139d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c09eab0 con 0x7f342c09f290 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.884+0000 7f343139d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f342c09f950 con 0x7f342c0a1f90 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.884+0000 7f34227fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f342c0a1f90 0x7f342c13a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.884+0000 7f342affd640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f342c09f290 0x7f342c0a1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42458/0 (socket says 172.21.15.130:42458) 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.884+0000 7f342affd640 1 -- 172.21.15.130:0/3496946360 learned_addr learned my addr 172.21.15.130:0/3496946360 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.885+0000 7f34227fc640 1 -- 172.21.15.130:0/3496946360 >> v1:172.21.15.130:6789/0 conn(0x7f342c09f290 legacy=0x7f342c0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.885+0000 7f34227fc640 1 -- 172.21.15.130:0/3496946360 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c09e7f0 con 0x7f342c0a1f90 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.885+0000 7f34227fc640 1 --2- 172.21.15.130:0/3496946360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c0a1f90 0x7f342c13a680 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00ae007f3cb9c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.885+0000 7f342a7fc640 1 -- 172.21.15.130:0/3496946360 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3414009d90 con 0x7f342c0a1f90 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.885+0000 7f342a7fc640 1 -- 172.21.15.130:0/3496946360 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f341400e230 con 0x7f342c0a1f90 2024-04-16T07:50:09.885 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.885+0000 7f342a7fc640 1 -- 172.21.15.130:0/3496946360 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f341400e4d0 con 0x7f342c0a1f90 2024-04-16T07:50:09.886 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 1 -- 172.21.15.130:0/3496946360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c0a1f90 msgr2=0x7f342c13a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 1 --2- 172.21.15.130:0/3496946360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c0a1f90 0x7f342c13a680 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 1 -- 172.21.15.130:0/3496946360 shutdown_connections 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 1 --2- 172.21.15.130:0/3496946360 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c0a1f90 0x7f342c13a680 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 1 -- 172.21.15.130:0/3496946360 >> 172.21.15.130:0/3496946360 conn(0x7f342c0a3b20 msgr2=0x7f342c0a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 1 -- 172.21.15.130:0/3496946360 shutdown_connections 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 1 -- 172.21.15.130:0/3496946360 wait complete. 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.886+0000 7f343139d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:09.887 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.887+0000 7f343139d640 1 Processor -- start 2024-04-16T07:50:09.888 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.888+0000 7f343139d640 1 -- start start 2024-04-16T07:50:09.889 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f343139d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c09f290 0x7f342c1ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:09.889 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f343139d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1aaf10 con 0x7f342c09f290 2024-04-16T07:50:09.889 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f342affd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c09f290 0x7f342c1ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:09.889 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f342affd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c09f290 0x7f342c1ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57996/0 (socket says 172.21.15.130:57996) 2024-04-16T07:50:09.889 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f342affd640 1 -- 172.21.15.130:0/1053330921 learned_addr learned my addr 172.21.15.130:0/1053330921 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:09.889 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f342affd640 1 -- 172.21.15.130:0/1053330921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1acce0 con 0x7f342c09f290 2024-04-16T07:50:09.890 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f342affd640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c09f290 0x7f342c1ad2f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:09.890 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f341c005a60 con 0x7f342c09f290 2024-04-16T07:50:09.890 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.889+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f341c005bc0 con 0x7f342c09f290 2024-04-16T07:50:09.890 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.890+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1ada10 con 0x7f342c09f290 2024-04-16T07:50:09.890 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.890+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f341c005e60 con 0x7f342c09f290 2024-04-16T07:50:09.890 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.890+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1acf10 con 0x7f342c09f290 2024-04-16T07:50:09.891 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.890+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 4) ==== 45778+0+0 (crc 0 0 0) 0x7f341c0070b0 con 0x7f342c09f290 2024-04-16T07:50:09.891 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.891+0000 7f3423fff640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f340c0388a0 0x7f340c03adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:09.891 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.891+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2319+0+0 (crc 0 0 0) 0x7f341c047ee0 con 0x7f342c09f290 2024-04-16T07:50:09.891 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.891+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c063930 con 0x7f342c09f290 2024-04-16T07:50:09.891 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.891+0000 7f34227fc640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f340c0388a0 0x7f340c03adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:09.891 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.891+0000 7f34227fc640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f340c0388a0 0x7f340c03adb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:09.896 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:09.896+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f341c01d040 con 0x7f342c09f290 2024-04-16T07:50:10.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.083+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f342c0739d0 con 0x7f342c09f290 2024-04-16T07:50:10.218 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.219+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f341c00c720 con 0x7f342c09f290 2024-04-16T07:50:10.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.221+0000 7f3423fff640 1 -- 172.21.15.130:0/1053330921 <== mon.0 v2:172.21.15.130:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v12) ==== 109+0+0 (crc 0 0 0) 0x7f341c00d330 con 0x7f342c09f290 2024-04-16T07:50:10.221 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'rbd' created 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f340c0388a0 msgr2=0x7f340c03adb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f340c0388a0 0x7f340c03adb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c09f290 msgr2=0x7f342c1ad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c09f290 0x7f342c1ad2f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 shutdown_connections 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f340c0388a0 0x7f340c03adb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 --2- 172.21.15.130:0/1053330921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f342c09f290 0x7f342c1ad2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:10.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 >> 172.21.15.130:0/1053330921 conn(0x7f342c0a3b20 msgr2=0x7f342c0a4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:10.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 shutdown_connections 2024-04-16T07:50:10.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.224+0000 7f343139d640 1 -- 172.21.15.130:0/1053330921 wait complete. 2024-04-16T07:50:10.239 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-16T07:50:10.369 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.360+0000 7efc2539d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:10.370 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.371+0000 7efc2539d640 1 Processor -- start 2024-04-16T07:50:10.370 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.371+0000 7efc2539d640 1 -- start start 2024-04-16T07:50:10.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc2539d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7efc20137df0 0x7efc2013a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:10.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc2539d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc20135a30 con 0x7efc20134f10 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc2539d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7efc200a11c0 con 0x7efc20137df0 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7efc20137df0 0x7efc2013a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1effd640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7efc20134f10 0x7efc20135320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42468/0 (socket says 172.21.15.130:42468) 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7efc20137df0 0x7efc2013a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58010/0 (socket says 172.21.15.130:58010) 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1effd640 1 -- 172.21.15.130:0/4019389662 learned_addr learned my addr 172.21.15.130:0/4019389662 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1e7fc640 1 -- 172.21.15.130:0/4019389662 >> v1:172.21.15.130:6789/0 conn(0x7efc20134f10 legacy=0x7efc20135320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1e7fc640 1 -- 172.21.15.130:0/4019389662 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2009af90 con 0x7efc20137df0 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1e7fc640 1 --2- 172.21.15.130:0/4019389662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20137df0 0x7efc2013a210 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9fdec8ff38f9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.372+0000 7efc1dffb640 1 -- 172.21.15.130:0/4019389662 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc08009d90 con 0x7efc20137df0 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.373+0000 7efc1dffb640 1 -- 172.21.15.130:0/4019389662 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc0800e230 con 0x7efc20137df0 2024-04-16T07:50:10.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.373+0000 7efc1dffb640 1 -- 172.21.15.130:0/4019389662 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc0800e4d0 con 0x7efc20137df0 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.373+0000 7efc2539d640 1 -- 172.21.15.130:0/4019389662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20137df0 msgr2=0x7efc2013a210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.373+0000 7efc2539d640 1 --2- 172.21.15.130:0/4019389662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20137df0 0x7efc2013a210 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.374+0000 7efc2539d640 1 -- 172.21.15.130:0/4019389662 shutdown_connections 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.374+0000 7efc2539d640 1 --2- 172.21.15.130:0/4019389662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20137df0 0x7efc2013a210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.374+0000 7efc2539d640 1 -- 172.21.15.130:0/4019389662 >> 172.21.15.130:0/4019389662 conn(0x7efc200a3a20 msgr2=0x7efc200a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.374+0000 7efc2539d640 1 -- 172.21.15.130:0/4019389662 shutdown_connections 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.374+0000 7efc2539d640 1 -- 172.21.15.130:0/4019389662 wait complete. 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.374+0000 7efc2539d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:10.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.375+0000 7efc2539d640 1 Processor -- start 2024-04-16T07:50:10.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.375+0000 7efc2539d640 1 -- start start 2024-04-16T07:50:10.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.376+0000 7efc2539d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20134f10 0x7efc2011e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:10.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.376+0000 7efc2539d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efc20121660 con 0x7efc20134f10 2024-04-16T07:50:10.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc1effd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20134f10 0x7efc2011e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:10.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc1effd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20134f10 0x7efc2011e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58020/0 (socket says 172.21.15.130:58020) 2024-04-16T07:50:10.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc1effd640 1 -- 172.21.15.130:0/2681811665 learned_addr learned my addr 172.21.15.130:0/2681811665 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:10.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc1effd640 1 -- 172.21.15.130:0/2681811665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2011fb60 con 0x7efc20134f10 2024-04-16T07:50:10.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc1effd640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20134f10 0x7efc2011e0c0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:10.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc177fe640 1 -- 172.21.15.130:0/2681811665 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc180070b0 con 0x7efc20134f10 2024-04-16T07:50:10.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc177fe640 1 -- 172.21.15.130:0/2681811665 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc18007210 con 0x7efc20134f10 2024-04-16T07:50:10.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc20121350 con 0x7efc20134f10 2024-04-16T07:50:10.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc177fe640 1 -- 172.21.15.130:0/2681811665 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc18007380 con 0x7efc20134f10 2024-04-16T07:50:10.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.377+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc20120ac0 con 0x7efc20134f10 2024-04-16T07:50:10.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.378+0000 7efc177fe640 1 -- 172.21.15.130:0/2681811665 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7efc18007580 con 0x7efc20134f10 2024-04-16T07:50:10.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.378+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc20063930 con 0x7efc20134f10 2024-04-16T07:50:10.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.379+0000 7efc177fe640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc00038930 0x7efc0003ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:10.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.379+0000 7efc177fe640 1 -- 172.21.15.130:0/2681811665 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2661+0+0 (crc 0 0 0) 0x7efc18048ae0 con 0x7efc20134f10 2024-04-16T07:50:10.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.379+0000 7efc1e7fc640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc00038930 0x7efc0003ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:10.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.379+0000 7efc1e7fc640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc00038930 0x7efc0003ae40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:10.384 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.384+0000 7efc177fe640 1 -- 172.21.15.130:0/2681811665 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7efc180092e0 con 0x7efc20134f10 2024-04-16T07:50:10.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:10.570+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7efc20135a30 con 0x7efc20134f10 2024-04-16T07:50:11.229 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.229+0000 7efc177fe640 1 -- 172.21.15.130:0/2681811665 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v13) ==== 173+0+0 (crc 0 0 0) 0x7efc20135a30 con 0x7efc20134f10 2024-04-16T07:50:11.229 INFO:teuthology.orchestra.run.smithi130.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc00038930 msgr2=0x7efc0003ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc00038930 0x7efc0003ae40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20134f10 msgr2=0x7efc2011e0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20134f10 0x7efc2011e0c0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 shutdown_connections 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc00038930 0x7efc0003ae40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 --2- 172.21.15.130:0/2681811665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc20134f10 0x7efc2011e0c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 >> 172.21.15.130:0/2681811665 conn(0x7efc200a3a20 msgr2=0x7efc201bcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:11.233 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 shutdown_connections 2024-04-16T07:50:11.233 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.232+0000 7efc2539d640 1 -- 172.21.15.130:0/2681811665 wait complete. 2024-04-16T07:50:11.250 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-16T07:50:11.251 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-16T07:50:11.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-16T07:50:11.251 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-16T07:50:11.251 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-16T07:50:11.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.378+0000 7f1d0c86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:11.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.382+0000 7f1d0c86d640 1 Processor -- start 2024-04-16T07:50:11.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.382+0000 7f1d0c86d640 1 -- start start 2024-04-16T07:50:11.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0c86d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f1d0809bde0 0x7f1d08136540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:11.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0c86d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d0809c710 con 0x7f1d0809b2c0 2024-04-16T07:50:11.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0c86d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f1d0806f940 con 0x7f1d0809bde0 2024-04-16T07:50:11.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d05d9b640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f1d0809bde0 0x7f1d08136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:11.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0659c640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f1d0809b2c0 0x7f1d0809b6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42478/0 (socket says 172.21.15.130:42478) 2024-04-16T07:50:11.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d05d9b640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f1d0809bde0 0x7f1d08136540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58036/0 (socket says 172.21.15.130:58036) 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0659c640 1 -- 172.21.15.130:0/3634253439 learned_addr learned my addr 172.21.15.130:0/3634253439 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d05d9b640 1 -- 172.21.15.130:0/3634253439 >> v1:172.21.15.130:6789/0 conn(0x7f1d0809b2c0 legacy=0x7f1d0809b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d05d9b640 1 -- 172.21.15.130:0/3634253439 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0806ecd0 con 0x7f1d0809bde0 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0559a640 1 -- 172.21.15.130:0/3634253439 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1742447850 0 0) 0x7f1d0809c710 con 0x7f1d0809b2c0 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d05d9b640 1 --2- 172.21.15.130:0/3634253439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809bde0 0x7f1d08136540 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8276ac2f706c23f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0559a640 1 -- 172.21.15.130:0/3634253439 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1cfc00ad20 con 0x7f1d0809bde0 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0559a640 1 -- 172.21.15.130:0/3634253439 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1cfc00ae80 con 0x7f1d0809bde0 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.384+0000 7f1d0559a640 1 -- 172.21.15.130:0/3634253439 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1cfc00f470 con 0x7f1d0809bde0 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.385+0000 7f1d0c86d640 1 -- 172.21.15.130:0/3634253439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809bde0 msgr2=0x7f1d08136540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.385+0000 7f1d0c86d640 1 --2- 172.21.15.130:0/3634253439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809bde0 0x7f1d08136540 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.385+0000 7f1d0c86d640 1 -- 172.21.15.130:0/3634253439 shutdown_connections 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.385+0000 7f1d0c86d640 1 --2- 172.21.15.130:0/3634253439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809bde0 0x7f1d08136540 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.385+0000 7f1d0c86d640 1 -- 172.21.15.130:0/3634253439 >> 172.21.15.130:0/3634253439 conn(0x7f1d080a0a30 msgr2=0x7f1d080a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:11.387 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.385+0000 7f1d0c86d640 1 -- 172.21.15.130:0/3634253439 shutdown_connections 2024-04-16T07:50:11.387 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.385+0000 7f1d0c86d640 1 -- 172.21.15.130:0/3634253439 wait complete. 2024-04-16T07:50:11.387 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.386+0000 7f1d0c86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:11.387 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.387+0000 7f1d0c86d640 1 Processor -- start 2024-04-16T07:50:11.387 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.387+0000 7f1d0c86d640 1 -- start start 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.388+0000 7f1d0c86d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809b2c0 0x7f1d081a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.388+0000 7f1d0c86d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081a4070 con 0x7f1d0809b2c0 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.388+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809b2c0 0x7f1d081a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.388+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809b2c0 0x7f1d081a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58052/0 (socket says 172.21.15.130:58052) 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.388+0000 7f1d0659c640 1 -- 172.21.15.130:0/815877790 learned_addr learned my addr 172.21.15.130:0/815877790 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.389+0000 7f1d0659c640 1 -- 172.21.15.130:0/815877790 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d081a2ab0 con 0x7f1d0809b2c0 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.389+0000 7f1d0659c640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809b2c0 0x7f1d081a0e90 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.389+0000 7f1cf6ffd640 1 -- 172.21.15.130:0/815877790 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1cf8004b80 con 0x7f1d0809b2c0 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.389+0000 7f1cf6ffd640 1 -- 172.21.15.130:0/815877790 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1cf8004ce0 con 0x7f1d0809b2c0 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.389+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0806cd10 con 0x7f1d0809b2c0 2024-04-16T07:50:11.389 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.389+0000 7f1cf6ffd640 1 -- 172.21.15.130:0/815877790 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1cf8007120 con 0x7f1d0809b2c0 2024-04-16T07:50:11.390 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.389+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d081a3390 con 0x7f1d0809b2c0 2024-04-16T07:50:11.390 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.390+0000 7f1cf6ffd640 1 -- 172.21.15.130:0/815877790 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f1cf80072d0 con 0x7f1d0809b2c0 2024-04-16T07:50:11.391 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.390+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d081a3c50 con 0x7f1d0809b2c0 2024-04-16T07:50:11.391 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.390+0000 7f1cf6ffd640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f1ce8038980 0x7f1ce803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:11.391 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.390+0000 7f1cf6ffd640 1 -- 172.21.15.130:0/815877790 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7f1cf8048310 con 0x7f1d0809b2c0 2024-04-16T07:50:11.391 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.391+0000 7f1d05d9b640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f1ce8038980 0x7f1ce803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:11.391 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.391+0000 7f1d05d9b640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f1ce8038980 0x7f1ce803ae90 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:11.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.395+0000 7f1cf6ffd640 1 -- 172.21.15.130:0/815877790 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f1cf800a9e0 con 0x7f1d0809b2c0 2024-04-16T07:50:11.618 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.618+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1d08063930 con 0x7f1d0809b2c0 2024-04-16T07:50:11.620 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.620+0000 7f1cf6ffd640 1 -- 172.21.15.130:0/815877790 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+87806 (crc 0 0 0) 0x7f1cf800a5d0 con 0x7f1d0809b2c0 2024-04-16T07:50:11.622 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f1ce8038980 msgr2=0x7f1ce803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f1ce8038980 0x7f1ce803ae90 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809b2c0 msgr2=0x7f1d081a0e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809b2c0 0x7f1d081a0e90 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 shutdown_connections 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f1ce8038980 0x7f1ce803ae90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 --2- 172.21.15.130:0/815877790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d0809b2c0 0x7f1d081a0e90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 >> 172.21.15.130:0/815877790 conn(0x7f1d080a0a30 msgr2=0x7f1d08070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:11.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 shutdown_connections 2024-04-16T07:50:11.624 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.623+0000 7f1d0c86d640 1 -- 172.21.15.130:0/815877790 wait complete. 2024-04-16T07:50:11.633 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4054838717},{"type":"v1","addr":"172.21.15.164:6801","nonce":4054838717}]},"active_addr":"172.21.15.164:6801/4054838717","active_change":"2024-04-16T07:50:04.157977+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3156651121}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":2770513891}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3494286696}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":1006833542}]}]} 2024-04-16T07:50:11.635 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-16T07:50:11.635 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-16T07:50:11.635 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T07:50:11.748 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.744+0000 7f96faaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:11.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.749+0000 7f96faaed640 1 Processor -- start 2024-04-16T07:50:11.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.749+0000 7f96faaed640 1 -- start start 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.750+0000 7f96faaed640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f96f409b5c0 0x7f96f409b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.750+0000 7f96faaed640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f409c310 con 0x7f96f4132a30 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.750+0000 7f96faaed640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f96f40710d0 con 0x7f96f409b5c0 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.750+0000 7f96f8889640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f96f409b5c0 0x7f96f409b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.750+0000 7f96f3fff640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f96f4132a30 0x7f96f4132e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42488/0 (socket says 172.21.15.130:42488) 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.750+0000 7f96f8889640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f96f409b5c0 0x7f96f409b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58066/0 (socket says 172.21.15.130:58066) 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.750+0000 7f96f3fff640 1 -- 172.21.15.130:0/1126291041 learned_addr learned my addr 172.21.15.130:0/1126291041 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.751+0000 7f96f8889640 1 -- 172.21.15.130:0/1126291041 >> v1:172.21.15.130:6789/0 conn(0x7f96f4132a30 legacy=0x7f96f4132e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.751+0000 7f96f8889640 1 -- 172.21.15.130:0/1126291041 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f409aed0 con 0x7f96f409b5c0 2024-04-16T07:50:11.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.751+0000 7f96f37fe640 1 -- 172.21.15.130:0/1126291041 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185868489 0 0) 0x7f96f409c310 con 0x7f96f4132a30 2024-04-16T07:50:11.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.751+0000 7f96f8889640 1 --2- 172.21.15.130:0/1126291041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f409b9d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3613cda5c42cff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:11.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.751+0000 7f96f37fe640 1 -- 172.21.15.130:0/1126291041 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f96e4009d80 con 0x7f96f409b5c0 2024-04-16T07:50:11.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.751+0000 7f96f37fe640 1 -- 172.21.15.130:0/1126291041 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f96e400e230 con 0x7f96f409b5c0 2024-04-16T07:50:11.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.751+0000 7f96f37fe640 1 -- 172.21.15.130:0/1126291041 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f96e400e4d0 con 0x7f96f409b5c0 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 1 -- 172.21.15.130:0/1126291041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 msgr2=0x7f96f409b9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 1 --2- 172.21.15.130:0/1126291041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f409b9d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 1 -- 172.21.15.130:0/1126291041 shutdown_connections 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 1 --2- 172.21.15.130:0/1126291041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f409b9d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 1 -- 172.21.15.130:0/1126291041 >> 172.21.15.130:0/1126291041 conn(0x7f96f40a2f50 msgr2=0x7f96f40a3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 1 -- 172.21.15.130:0/1126291041 shutdown_connections 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 1 -- 172.21.15.130:0/1126291041 wait complete. 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.752+0000 7f96faaed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.753+0000 7f96faaed640 1 Processor -- start 2024-04-16T07:50:11.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.753+0000 7f96faaed640 1 -- start start 2024-04-16T07:50:11.755 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96faaed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:11.755 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96faaed640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f96f40702a0 con 0x7f96f409b5c0 2024-04-16T07:50:11.755 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f8889640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:11.755 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f8889640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f406ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58076/0 (socket says 172.21.15.130:58076) 2024-04-16T07:50:11.755 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f8889640 1 -- 172.21.15.130:0/2929449353 learned_addr learned my addr 172.21.15.130:0/2929449353 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:11.755 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f8889640 1 -- 172.21.15.130:0/2929449353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f4070a30 con 0x7f96f409b5c0 2024-04-16T07:50:11.756 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f8889640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f406ee50 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:11.756 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f17fa640 1 -- 172.21.15.130:0/2929449353 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f96e4013740 con 0x7f96f409b5c0 2024-04-16T07:50:11.756 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f17fa640 1 -- 172.21.15.130:0/2929449353 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f96e40138a0 con 0x7f96f409b5c0 2024-04-16T07:50:11.756 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f4122080 con 0x7f96f409b5c0 2024-04-16T07:50:11.756 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96f17fa640 1 -- 172.21.15.130:0/2929449353 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f96e4013b40 con 0x7f96f409b5c0 2024-04-16T07:50:11.756 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.755+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f4120b60 con 0x7f96f409b5c0 2024-04-16T07:50:11.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.756+0000 7f96f17fa640 1 -- 172.21.15.130:0/2929449353 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f96e4013cf0 con 0x7f96f409b5c0 2024-04-16T07:50:11.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.756+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f4121920 con 0x7f96f409b5c0 2024-04-16T07:50:11.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.757+0000 7f96f17fa640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f96e00389b0 0x7f96e003aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:11.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.757+0000 7f96f17fa640 1 -- 172.21.15.130:0/2929449353 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7f96e4018040 con 0x7f96f409b5c0 2024-04-16T07:50:11.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.757+0000 7f96f3fff640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f96e00389b0 0x7f96e003aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:11.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.757+0000 7f96f3fff640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f96e00389b0 0x7f96e003aec0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:11.762 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.762+0000 7f96f17fa640 1 -- 172.21.15.130:0/2929449353 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f96e4018e10 con 0x7f96f409b5c0 2024-04-16T07:50:11.938 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.938+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96f4063930 con 0x7f96f409b5c0 2024-04-16T07:50:11.938 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.939+0000 7f96f17fa640 1 -- 172.21.15.130:0/2929449353 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+8755 (crc 0 0 0) 0x7f96e4007e30 con 0x7f96f409b5c0 2024-04-16T07:50:11.939 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:50:11.939 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":13,"fsid":"06ce95cb-daea-4a00-b123-d08aef1a06ab","created":"2024-04-16T07:49:59.765016+0000","modified":"2024-04-16T07:50:11.222780+0000","last_up_change":"2024-04-16T07:50:06.203593+0000","last_in_change":"2024-04-16T07:50:01.924952+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T07:50:08.183717+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-16T07:50:10.087130+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3361768573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6801","nonce":3361768573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":3361768573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6809","nonce":3361768573}]},"public_addr":"172.21.15.130:6802/3361768573","cluster_addr":"172.21.15.130:6801/3361768573","heartbeat_back_addr":"172.21.15.130:6808/3361768573","heartbeat_front_addr":"172.21.15.130:6809/3361768573","state":["exists","up"]},{"osd":1,"uuid":"1935374e-a32a-4cda-95d3-af6a8e92f6d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3830191025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6805","nonce":3830191025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3830191025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6803","nonce":3830191025}]},"public_addr":"172.21.15.130:6804/3830191025","cluster_addr":"172.21.15.130:6805/3830191025","heartbeat_back_addr":"172.21.15.130:6806/3830191025","heartbeat_front_addr":"172.21.15.130:6803/3830191025","state":["exists","up"]},{"osd":2,"uuid":"0e51d8b6-100a-42fa-b7cf-9d57d230ee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2835634474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6805","nonce":2835634474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":2835634474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2835634474}]},"public_addr":"172.21.15.164:6804/2835634474","cluster_addr":"172.21.15.164:6805/2835634474","heartbeat_back_addr":"172.21.15.164:6815/2835634474","heartbeat_front_addr":"172.21.15.164:6816/2835634474","state":["exists","up"]},{"osd":3,"uuid":"9bc937c4-1eec-4856-965b-3de26632e115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3962925382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6803","nonce":3962925382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6807","nonce":3962925382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3962925382}]},"public_addr":"172.21.15.164:6806/3962925382","cluster_addr":"172.21.15.164:6803/3962925382","heartbeat_back_addr":"172.21.15.164:6807/3962925382","heartbeat_front_addr":"172.21.15.164:6802/3962925382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T07:50:11.941 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f96e00389b0 msgr2=0x7f96e003aec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.941 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f96e00389b0 0x7f96e003aec0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 msgr2=0x7f96f406ee50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f406ee50 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 shutdown_connections 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f96e00389b0 0x7f96e003aec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 --2- 172.21.15.130:0/2929449353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96f409b5c0 0x7f96f406ee50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 >> 172.21.15.130:0/2929449353 conn(0x7f96f40a2f50 msgr2=0x7f96f41baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 shutdown_connections 2024-04-16T07:50:11.942 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:11.941+0000 7f96faaed640 1 -- 172.21.15.130:0/2929449353 wait complete. 2024-04-16T07:50:11.952 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-16T07:50:11.952 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T07:50:12.064 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.060+0000 7f9a3b15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.065 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.065+0000 7f9a3b15d640 1 Processor -- start 2024-04-16T07:50:12.065 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.065+0000 7f9a3b15d640 1 -- start start 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.066+0000 7f9a3b15d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f9a3409bde0 0x7f9a34136540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.066+0000 7f9a3b15d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a3409c710 con 0x7f9a3409b2c0 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.066+0000 7f9a3b15d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f9a3406f940 con 0x7f9a3409bde0 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.066+0000 7f9a33fff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f9a3409bde0 0x7f9a34136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.066+0000 7f9a33fff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f9a3409bde0 0x7f9a34136540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58078/0 (socket says 172.21.15.130:58078) 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.066+0000 7f9a38ef9640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f9a3409b2c0 0x7f9a3409b6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42502/0 (socket says 172.21.15.130:42502) 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.066+0000 7f9a33fff640 1 -- 172.21.15.130:0/944180594 learned_addr learned my addr 172.21.15.130:0/944180594 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.067+0000 7f9a33fff640 1 -- 172.21.15.130:0/944180594 >> v1:172.21.15.130:6789/0 conn(0x7f9a3409b2c0 legacy=0x7f9a3409b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.068 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.067+0000 7f9a33fff640 1 -- 172.21.15.130:0/944180594 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3406ecd0 con 0x7f9a3409bde0 2024-04-16T07:50:12.068 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.067+0000 7f9a33fff640 1 --2- 172.21.15.130:0/944180594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409bde0 0x7f9a34136540 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966bb07f9674a931 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.068 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.067+0000 7f9a337fe640 1 -- 172.21.15.130:0/944180594 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a2400e040 con 0x7f9a3409bde0 2024-04-16T07:50:12.068 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.067+0000 7f9a337fe640 1 -- 172.21.15.130:0/944180594 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9a240091e0 con 0x7f9a3409bde0 2024-04-16T07:50:12.068 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.067+0000 7f9a337fe640 1 -- 172.21.15.130:0/944180594 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a24007730 con 0x7f9a3409bde0 2024-04-16T07:50:12.069 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 1 -- 172.21.15.130:0/944180594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409bde0 msgr2=0x7f9a34136540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.069 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 1 --2- 172.21.15.130:0/944180594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409bde0 0x7f9a34136540 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.069 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 1 -- 172.21.15.130:0/944180594 shutdown_connections 2024-04-16T07:50:12.069 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 1 --2- 172.21.15.130:0/944180594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409bde0 0x7f9a34136540 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.070 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 1 -- 172.21.15.130:0/944180594 >> 172.21.15.130:0/944180594 conn(0x7f9a340a0a30 msgr2=0x7f9a340a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.070 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 1 -- 172.21.15.130:0/944180594 shutdown_connections 2024-04-16T07:50:12.070 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 1 -- 172.21.15.130:0/944180594 wait complete. 2024-04-16T07:50:12.070 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.068+0000 7f9a3b15d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.070 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.069+0000 7f9a3b15d640 1 Processor -- start 2024-04-16T07:50:12.070 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.069+0000 7f9a3b15d640 1 -- start start 2024-04-16T07:50:12.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a3b15d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409b2c0 0x7f9a341206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a3b15d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9a341248e0 con 0x7f9a3409b2c0 2024-04-16T07:50:12.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a38ef9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409b2c0 0x7f9a341206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a38ef9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409b2c0 0x7f9a341206e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58090/0 (socket says 172.21.15.130:58090) 2024-04-16T07:50:12.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a38ef9640 1 -- 172.21.15.130:0/2542754941 learned_addr learned my addr 172.21.15.130:0/2542754941 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a38ef9640 1 -- 172.21.15.130:0/2542754941 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a34123520 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a38ef9640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409b2c0 0x7f9a341206e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a317fa640 1 -- 172.21.15.130:0/2542754941 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a1c0059a0 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a317fa640 1 -- 172.21.15.130:0/2542754941 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9a1c005b00 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a34121900 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a317fa640 1 -- 172.21.15.130:0/2542754941 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9a1c005da0 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.071+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a34123c70 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.072+0000 7f9a317fa640 1 -- 172.21.15.130:0/2542754941 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f9a1c0070b0 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.072+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a34123000 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.072+0000 7f9a317fa640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f9a140389b0 0x7f9a1403aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.073+0000 7f9a317fa640 1 -- 172.21.15.130:0/2542754941 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2672+0+0 (crc 0 0 0) 0x7f9a1c0480a0 con 0x7f9a3409b2c0 2024-04-16T07:50:12.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.073+0000 7f9a33fff640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f9a140389b0 0x7f9a1403aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.073+0000 7f9a33fff640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f9a140389b0 0x7f9a1403aec0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.078+0000 7f9a317fa640 1 -- 172.21.15.130:0/2542754941 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f9a1c00d330 con 0x7f9a3409b2c0 2024-04-16T07:50:12.257 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.257+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a34063930 con 0x7f9a3409b2c0 2024-04-16T07:50:12.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.258+0000 7f9a317fa640 1 -- 172.21.15.130:0/2542754941 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8746 (crc 0 0 0) 0x7f9a1c00d150 con 0x7f9a3409b2c0 2024-04-16T07:50:12.258 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:50:12.258 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":14,"fsid":"06ce95cb-daea-4a00-b123-d08aef1a06ab","created":"2024-04-16T07:49:59.765016+0000","modified":"2024-04-16T07:50:12.230190+0000","last_up_change":"2024-04-16T07:50:06.203593+0000","last_in_change":"2024-04-16T07:50:01.924952+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T07:50:08.183717+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-16T07:50:10.087130+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ab0da52-ec9e-4b3b-bc1e-1a8c6a75449a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3361768573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6801","nonce":3361768573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":3361768573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6809","nonce":3361768573}]},"public_addr":"172.21.15.130:6802/3361768573","cluster_addr":"172.21.15.130:6801/3361768573","heartbeat_back_addr":"172.21.15.130:6808/3361768573","heartbeat_front_addr":"172.21.15.130:6809/3361768573","state":["exists","up"]},{"osd":1,"uuid":"1935374e-a32a-4cda-95d3-af6a8e92f6d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3830191025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6805","nonce":3830191025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3830191025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6803","nonce":3830191025}]},"public_addr":"172.21.15.130:6804/3830191025","cluster_addr":"172.21.15.130:6805/3830191025","heartbeat_back_addr":"172.21.15.130:6806/3830191025","heartbeat_front_addr":"172.21.15.130:6803/3830191025","state":["exists","up"]},{"osd":2,"uuid":"0e51d8b6-100a-42fa-b7cf-9d57d230ee1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2835634474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6805","nonce":2835634474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":2835634474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2835634474}]},"public_addr":"172.21.15.164:6804/2835634474","cluster_addr":"172.21.15.164:6805/2835634474","heartbeat_back_addr":"172.21.15.164:6815/2835634474","heartbeat_front_addr":"172.21.15.164:6816/2835634474","state":["exists","up"]},{"osd":3,"uuid":"9bc937c4-1eec-4856-965b-3de26632e115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3962925382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6803","nonce":3962925382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6807","nonce":3962925382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3962925382}]},"public_addr":"172.21.15.164:6806/3962925382","cluster_addr":"172.21.15.164:6803/3962925382","heartbeat_back_addr":"172.21.15.164:6807/3962925382","heartbeat_front_addr":"172.21.15.164:6802/3962925382","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T07:50:12.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f9a140389b0 msgr2=0x7f9a1403aec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f9a140389b0 0x7f9a1403aec0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409b2c0 msgr2=0x7f9a341206e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409b2c0 0x7f9a341206e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 shutdown_connections 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f9a140389b0 0x7f9a1403aec0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 --2- 172.21.15.130:0/2542754941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a3409b2c0 0x7f9a341206e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 >> 172.21.15.130:0/2542754941 conn(0x7f9a340a0a30 msgr2=0x7f9a340726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 shutdown_connections 2024-04-16T07:50:12.261 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.260+0000 7f9a3b15d640 1 -- 172.21.15.130:0/2542754941 wait complete. 2024-04-16T07:50:12.272 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-16T07:50:12.272 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-16T07:50:12.273 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-16T07:50:12.273 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-16T07:50:12.397 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.393+0000 7f0b38c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.397 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.393+0000 7f07fb8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.398 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.393+0000 7f7d4419d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.398 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.393+0000 7f2163f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.398 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.398+0000 7f07fb8ad640 1 Processor -- start 2024-04-16T07:50:12.399 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.398+0000 7f7d4419d640 1 Processor -- start 2024-04-16T07:50:12.399 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.398+0000 7f07fb8ad640 1 -- start start 2024-04-16T07:50:12.399 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.398+0000 7f0b38c6d640 1 Processor -- start 2024-04-16T07:50:12.400 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.399+0000 7f7d4419d640 1 -- start start 2024-04-16T07:50:12.400 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.399+0000 7f2163f9d640 1 Processor -- start 2024-04-16T07:50:12.401 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.399+0000 7f0b38c6d640 1 -- start start 2024-04-16T07:50:12.401 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.399+0000 7f2163f9d640 1 -- start start 2024-04-16T07:50:12.401 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07fb8ad640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f07f40a36c0 0x7f07f412dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.402 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d4419d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f7d3c132240 0x7f7d3c139e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.403 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d4419d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3c1315f0 con 0x7f7d3c09bae0 2024-04-16T07:50:12.403 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d4419d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c09c640 con 0x7f7d3c132240 2024-04-16T07:50:12.403 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d41f39640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f7d3c09bae0 0x7f7d3c131b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42516/0 (socket says 172.21.15.130:42516) 2024-04-16T07:50:12.403 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d41f39640 1 -- 172.21.15.130:0/2464598595 learned_addr learned my addr 172.21.15.130:0/2464598595 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:12.403 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d41738640 1 --2- 172.21.15.130:0/2464598595 >> v2:172.21.15.130:3300/0 conn(0x7f7d3c132240 0x7f7d3c139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.403 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07fb8ad640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f406e7b0 con 0x7f07f412e530 2024-04-16T07:50:12.403 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07fb8ad640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f07f412e940 con 0x7f07f40a36c0 2024-04-16T07:50:12.404 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07f8e48640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f07f412e530 0x7f07f4130f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42526/0 (socket says 172.21.15.130:42526) 2024-04-16T07:50:12.404 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07f8e48640 1 -- 172.21.15.130:0/60393740 learned_addr learned my addr 172.21.15.130:0/60393740 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:12.404 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07f9649640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f07f40a36c0 0x7f07f412dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.404 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d41738640 1 -- 172.21.15.130:0/2464598595 >> v1:172.21.15.130:6789/0 conn(0x7f7d3c09bae0 legacy=0x7f7d3c131b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.404 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d41738640 1 -- 172.21.15.130:0/2464598595 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c09b030 con 0x7f7d3c132240 2024-04-16T07:50:12.405 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07f9649640 1 -- 172.21.15.130:0/60393740 >> v1:172.21.15.130:6789/0 conn(0x7f07f412e530 legacy=0x7f07f4130f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.405 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f07f9649640 1 -- 172.21.15.130:0/60393740 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f412da10 con 0x7f07f40a36c0 2024-04-16T07:50:12.405 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.400+0000 7f7d41738640 1 --2- 172.21.15.130:0/2464598595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c132240 0x7f7d3c139e50 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ff016fb7741c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.405 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f0b38c6d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f0b34132240 0x7f0b34139e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.406 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f07f9649640 1 --2- 172.21.15.130:0/60393740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f412dff0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cbfe7f3e33223d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.406 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f7d40f37640 1 -- 172.21.15.130:0/2464598595 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d2c009d90 con 0x7f7d3c132240 2024-04-16T07:50:12.406 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f0b3259c640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f0b3409bae0 0x7f0b34131b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42542/0 (socket says 172.21.15.130:42542) 2024-04-16T07:50:12.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2163f9d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f215c132240 0x7f215c139e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2163f9d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215c1315f0 con 0x7f215c09bae0 2024-04-16T07:50:12.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2163f9d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f215c09c640 con 0x7f215c132240 2024-04-16T07:50:12.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f7d40f37640 1 -- 172.21.15.130:0/2464598595 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7d2c00e230 con 0x7f7d3c132240 2024-04-16T07:50:12.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f7d40f37640 1 -- 172.21.15.130:0/2464598595 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d2c00e4d0 con 0x7f7d3c132240 2024-04-16T07:50:12.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f07ebfff640 1 -- 172.21.15.130:0/60393740 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07f0012040 con 0x7f07f40a36c0 2024-04-16T07:50:12.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f0b3259c640 1 -- 172.21.15.130:0/2481068601 learned_addr learned my addr 172.21.15.130:0/2481068601 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:12.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f07ebfff640 1 -- 172.21.15.130:0/60393740 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f07f000d0b0 con 0x7f07f40a36c0 2024-04-16T07:50:12.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f07ebfff640 1 -- 172.21.15.130:0/60393740 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07f000b660 con 0x7f07f40a36c0 2024-04-16T07:50:12.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2161d39640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f215c09bae0 0x7f215c131b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42548/0 (socket says 172.21.15.130:42548) 2024-04-16T07:50:12.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2161d39640 1 -- 172.21.15.130:0/1079527651 learned_addr learned my addr 172.21.15.130:0/1079527651 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:12.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f0b38c6d640 1 -- 172.21.15.130:0/2481068601 --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b341315f0 con 0x7f0b3409bae0 2024-04-16T07:50:12.410 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2161538640 1 --2- 172.21.15.130:0/1079527651 >> v2:172.21.15.130:3300/0 conn(0x7f215c132240 0x7f215c139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.410 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f0b38c6d640 1 -- 172.21.15.130:0/2481068601 --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3409c640 con 0x7f0b34132240 2024-04-16T07:50:12.410 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f0b2bfff640 1 --2- 172.21.15.130:0/2481068601 >> v2:172.21.15.130:3300/0 conn(0x7f0b34132240 0x7f0b34139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2160d37640 1 -- 172.21.15.130:0/1079527651 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1989107072 0 0) 0x7f215c1315f0 con 0x7f215c09bae0 2024-04-16T07:50:12.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2160d37640 1 -- 172.21.15.130:0/1079527651 --> v1:172.21.15.130:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2148003580 con 0x7f215c09bae0 2024-04-16T07:50:12.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2161538640 1 -- 172.21.15.130:0/1079527651 >> v1:172.21.15.130:6789/0 conn(0x7f215c09bae0 legacy=0x7f215c131b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.401+0000 7f2161538640 1 -- 172.21.15.130:0/1079527651 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c09b030 con 0x7f215c132240 2024-04-16T07:50:12.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f2161538640 1 --2- 172.21.15.130:0/1079527651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c132240 0x7f215c139e50 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467c5c91035243d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f0b2bfff640 1 -- 172.21.15.130:0/2481068601 >> v1:172.21.15.130:6789/0 conn(0x7f0b3409bae0 legacy=0x7f0b34131b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f0b2bfff640 1 -- 172.21.15.130:0/2481068601 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3409b030 con 0x7f0b34132240 2024-04-16T07:50:12.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f2160d37640 1 -- 172.21.15.130:0/1079527651 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f214c009d90 con 0x7f215c132240 2024-04-16T07:50:12.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f2160d37640 1 -- 172.21.15.130:0/1079527651 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f214c00e230 con 0x7f215c132240 2024-04-16T07:50:12.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f2160d37640 1 -- 172.21.15.130:0/1079527651 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f214c00e4d0 con 0x7f215c132240 2024-04-16T07:50:12.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f0b2bfff640 1 --2- 172.21.15.130:0/2481068601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b34132240 0x7f0b34139e50 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106947589df13b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f7d4419d640 1 -- 172.21.15.130:0/2464598595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c132240 msgr2=0x7f7d3c139e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f0b31d9b640 1 -- 172.21.15.130:0/2481068601 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0b1c009d90 con 0x7f0b34132240 2024-04-16T07:50:12.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f0b31d9b640 1 -- 172.21.15.130:0/2481068601 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0b1c00e230 con 0x7f0b34132240 2024-04-16T07:50:12.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f0b31d9b640 1 -- 172.21.15.130:0/2481068601 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0b1c00e4d0 con 0x7f0b34132240 2024-04-16T07:50:12.414 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f7d4419d640 1 --2- 172.21.15.130:0/2464598595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c132240 0x7f7d3c139e50 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.414 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f07fb8ad640 1 -- 172.21.15.130:0/60393740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 msgr2=0x7f07f412dff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.414 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/60393740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f412dff0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.415 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f7d4419d640 1 -- 172.21.15.130:0/2464598595 shutdown_connections 2024-04-16T07:50:12.415 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f7d4419d640 1 --2- 172.21.15.130:0/2464598595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c132240 0x7f7d3c139e50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.415 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f7d4419d640 1 -- 172.21.15.130:0/2464598595 >> 172.21.15.130:0/2464598595 conn(0x7f7d3c0a3c20 msgr2=0x7f7d3c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.415 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f07fb8ad640 1 -- 172.21.15.130:0/60393740 shutdown_connections 2024-04-16T07:50:12.415 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/60393740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f412dff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.415 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f07fb8ad640 1 -- 172.21.15.130:0/60393740 >> 172.21.15.130:0/60393740 conn(0x7f07f40a2ba0 msgr2=0x7f07f40a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.416 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f7d4419d640 1 -- 172.21.15.130:0/2464598595 shutdown_connections 2024-04-16T07:50:12.416 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f7d4419d640 1 -- 172.21.15.130:0/2464598595 wait complete. 2024-04-16T07:50:12.416 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f07fb8ad640 1 -- 172.21.15.130:0/60393740 shutdown_connections 2024-04-16T07:50:12.416 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.402+0000 7f07fb8ad640 1 -- 172.21.15.130:0/60393740 wait complete. 2024-04-16T07:50:12.416 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f7d4419d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.417 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f07fb8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.417 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f2163f9d640 1 -- 172.21.15.130:0/1079527651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c132240 msgr2=0x7f215c139e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.417 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f2163f9d640 1 --2- 172.21.15.130:0/1079527651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c132240 0x7f215c139e50 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.417 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f2163f9d640 1 -- 172.21.15.130:0/1079527651 shutdown_connections 2024-04-16T07:50:12.417 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f2163f9d640 1 --2- 172.21.15.130:0/1079527651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c132240 0x7f215c139e50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.417 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f2163f9d640 1 -- 172.21.15.130:0/1079527651 >> 172.21.15.130:0/1079527651 conn(0x7f215c0a3c20 msgr2=0x7f215c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.418 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f0b38c6d640 1 -- 172.21.15.130:0/2481068601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b34132240 msgr2=0x7f0b34139e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.418 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f2163f9d640 1 -- 172.21.15.130:0/1079527651 shutdown_connections 2024-04-16T07:50:12.418 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/2481068601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b34132240 0x7f0b34139e50 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.403+0000 7f2163f9d640 1 -- 172.21.15.130:0/1079527651 wait complete. 2024-04-16T07:50:12.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f0b38c6d640 1 -- 172.21.15.130:0/2481068601 shutdown_connections 2024-04-16T07:50:12.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/2481068601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b34132240 0x7f0b34139e50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f0b38c6d640 1 -- 172.21.15.130:0/2481068601 >> 172.21.15.130:0/2481068601 conn(0x7f0b340a3c20 msgr2=0x7f0b340a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f0b38c6d640 1 -- 172.21.15.130:0/2481068601 shutdown_connections 2024-04-16T07:50:12.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f0b38c6d640 1 -- 172.21.15.130:0/2481068601 wait complete. 2024-04-16T07:50:12.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f2163f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f0b38c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f7d4419d640 1 Processor -- start 2024-04-16T07:50:12.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f07fb8ad640 1 Processor -- start 2024-04-16T07:50:12.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f07fb8ad640 1 -- start start 2024-04-16T07:50:12.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.404+0000 7f7d4419d640 1 -- start start 2024-04-16T07:50:12.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f2163f9d640 1 Processor -- start 2024-04-16T07:50:12.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f2163f9d640 1 -- start start 2024-04-16T07:50:12.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f7d4419d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c09bae0 0x7f7d3c11d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f7d4419d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c121070 con 0x7f7d3c09bae0 2024-04-16T07:50:12.423 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f07fb8ad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f411a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.423 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f07fb8ad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f07f411bdb0 con 0x7f07f40a36c0 2024-04-16T07:50:12.423 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f0b38c6d640 1 Processor -- start 2024-04-16T07:50:12.423 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.405+0000 7f7d41f39640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c09bae0 0x7f7d3c11d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.424 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d41f39640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c09bae0 0x7f7d3c11d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58130/0 (socket says 172.21.15.130:58130) 2024-04-16T07:50:12.424 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d41f39640 1 -- 172.21.15.130:0/407131219 learned_addr learned my addr 172.21.15.130:0/407131219 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.424 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f0b38c6d640 1 -- start start 2024-04-16T07:50:12.424 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f07f9649640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f411a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.424 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f07f9649640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f411a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58146/0 (socket says 172.21.15.130:58146) 2024-04-16T07:50:12.424 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f07f9649640 1 -- 172.21.15.130:0/4190286811 learned_addr learned my addr 172.21.15.130:0/4190286811 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.425 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d41f39640 1 -- 172.21.15.130:0/407131219 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c11fb40 con 0x7f7d3c09bae0 2024-04-16T07:50:12.425 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f07f9649640 1 -- 172.21.15.130:0/4190286811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f4119d60 con 0x7f07f40a36c0 2024-04-16T07:50:12.425 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d41f39640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c09bae0 0x7f7d3c11d3d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.426 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f07f9649640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f411a290 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.426 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d38005a60 con 0x7f7d3c09bae0 2024-04-16T07:50:12.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c1313d0 con 0x7f7d3c09bae0 2024-04-16T07:50:12.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c121330 con 0x7f7d3c09bae0 2024-04-16T07:50:12.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07f0012040 con 0x7f07f40a36c0 2024-04-16T07:50:12.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f07f0013040 con 0x7f07f40a36c0 2024-04-16T07:50:12.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f411b4d0 con 0x7f07f40a36c0 2024-04-16T07:50:12.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7d38005bc0 con 0x7f7d3c09bae0 2024-04-16T07:50:12.427 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.406+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d38005a60 con 0x7f7d3c09bae0 2024-04-16T07:50:12.428 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f411c0d0 con 0x7f07f40a36c0 2024-04-16T07:50:12.428 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f2163f9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c09bae0 0x7f215c11d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.428 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f2163f9d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f215c06e920 con 0x7f215c09bae0 2024-04-16T07:50:12.428 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f0b38c6d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b3409bae0 0x7f0b341c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d24000f80 con 0x7f7d3c09bae0 2024-04-16T07:50:12.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f0b38c6d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341216d0 con 0x7f0b3409bae0 2024-04-16T07:50:12.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.407+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f07f00156a0 con 0x7f07f40a36c0 2024-04-16T07:50:12.429 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f2161d39640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c09bae0 0x7f215c11d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.430 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f0b3259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b3409bae0 0x7f0b341c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.430 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f0b3259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b3409bae0 0x7f0b341c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58162/0 (socket says 172.21.15.130:58162) 2024-04-16T07:50:12.430 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f0b3259c640 1 -- 172.21.15.130:0/194416249 learned_addr learned my addr 172.21.15.130:0/194416249 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.430 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f2161d39640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c09bae0 0x7f215c11d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58156/0 (socket says 172.21.15.130:58156) 2024-04-16T07:50:12.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f2161d39640 1 -- 172.21.15.130:0/3810228214 learned_addr learned my addr 172.21.15.130:0/3810228214 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07f4119bf0 con 0x7f07f40a36c0 2024-04-16T07:50:12.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f0b3259c640 1 -- 172.21.15.130:0/194416249 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3406e990 con 0x7f0b3409bae0 2024-04-16T07:50:12.431 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f7d3800c780 con 0x7f7d3c09bae0 2024-04-16T07:50:12.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f7d327fc640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d200389c0 0x7f7d2003aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f7d38048160 con 0x7f7d3c09bae0 2024-04-16T07:50:12.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f7d327fc640 1 --2- 172.21.15.130:0/407131219 >> v2:172.21.15.164:6806/3962925382 conn(0x7f7d2003d9c0 0x7f7d2003fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 --> v2:172.21.15.164:6806/3962925382 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d200404e0 con 0x7f7d2003d9c0 2024-04-16T07:50:12.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.408+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f7d380484f0 con 0x7f7d3c09bae0 2024-04-16T07:50:12.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f2161d39640 1 -- 172.21.15.130:0/3810228214 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c1212d0 con 0x7f215c09bae0 2024-04-16T07:50:12.432 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f0b3259c640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b3409bae0 0x7f0b341c9c30 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.433 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f07f0015800 con 0x7f07f40a36c0 2024-04-16T07:50:12.433 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f7d41738640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d200389c0 0x7f7d2003aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.433 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f2161d39640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c09bae0 0x7f215c11d3d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f07e9ffb640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f07d8038a20 0x7f07d803af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f07f004d370 con 0x7f07f40a36c0 2024-04-16T07:50:12.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0b24005a60 con 0x7f0b3409bae0 2024-04-16T07:50:12.434 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f07e9ffb640 1 --2- 172.21.15.130:0/4190286811 >> v2:172.21.15.130:6802/3361768573 conn(0x7f07d803daa0 0x7f07d803ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.435 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0b24005bc0 con 0x7f0b3409bae0 2024-04-16T07:50:12.435 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2158005a60 con 0x7f215c09bae0 2024-04-16T07:50:12.435 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 --> v2:172.21.15.130:6802/3361768573 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07d80405c0 con 0x7f07d803daa0 2024-04-16T07:50:12.435 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f07f004d680 con 0x7f07f40a36c0 2024-04-16T07:50:12.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0b24005e60 con 0x7f0b3409bae0 2024-04-16T07:50:12.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b34121ed0 con 0x7f0b3409bae0 2024-04-16T07:50:12.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b34124350 con 0x7f0b3409bae0 2024-04-16T07:50:12.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f07f8e48640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f07d8038a20 0x7f07d803af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2158005bc0 con 0x7f215c09bae0 2024-04-16T07:50:12.436 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2158005e60 con 0x7f215c09bae0 2024-04-16T07:50:12.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c11efd0 con 0x7f215c09bae0 2024-04-16T07:50:12.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f7d41738640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d200389c0 0x7f7d2003aed0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.410+0000 7f7d4273a640 1 --2- 172.21.15.130:0/407131219 >> v2:172.21.15.164:6806/3962925382 conn(0x7f7d2003d9c0 0x7f7d2003fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-16T07:50:12.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.410+0000 7f07f8e48640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f07d8038a20 0x7f07d803af30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.437 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.409+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c131190 con 0x7f215c09bae0 2024-04-16T07:50:12.438 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.411+0000 7f07f9e4a640 1 --2- 172.21.15.130:0/4190286811 >> v2:172.21.15.130:6802/3361768573 conn(0x7f07d803daa0 0x7f07d803ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-16T07:50:12.438 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.411+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f0b2400c780 con 0x7f0b3409bae0 2024-04-16T07:50:12.438 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.411+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b3409b440 con 0x7f0b3409bae0 2024-04-16T07:50:12.438 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.411+0000 7f0b2affd640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0b140389d0 0x7f0b1403aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.439 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.411+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f215c09b440 con 0x7f215c09bae0 2024-04-16T07:50:12.439 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.411+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f0b24048050 con 0x7f0b3409bae0 2024-04-16T07:50:12.439 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f0b2affd640 1 --2- 172.21.15.130:0/194416249 >> v2:172.21.15.130:6804/3830191025 conn(0x7f0b1403d9d0 0x7f0b1403fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.439 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 --> v2:172.21.15.130:6804/3830191025 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b140404f0 con 0x7f0b1403d9d0 2024-04-16T07:50:12.439 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f0b24048390 con 0x7f0b3409bae0 2024-04-16T07:50:12.439 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f0b2bfff640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0b140389d0 0x7f0b1403aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.440 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f21580070b0 con 0x7f215c09bae0 2024-04-16T07:50:12.440 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f21527fc640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f21400389d0 0x7f214003aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.440 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f2158048260 con 0x7f215c09bae0 2024-04-16T07:50:12.440 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f21527fc640 1 --2- 172.21.15.130:0/3810228214 >> v2:172.21.15.164:6804/2835634474 conn(0x7f214003d9d0 0x7f214003fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.440 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 --> v2:172.21.15.164:6804/2835634474 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21400404f0 con 0x7f214003d9d0 2024-04-16T07:50:12.441 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (crc 0 0 0) 0x7f215804a680 con 0x7f215c09bae0 2024-04-16T07:50:12.441 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f2161538640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f21400389d0 0x7f214003aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.441 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.412+0000 7f0b2bfff640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0b140389d0 0x7f0b1403aee0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.441 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.413+0000 7f2161538640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f21400389d0 0x7f214003aee0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.441 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.413+0000 7f0b32d9d640 1 --2- 172.21.15.130:0/194416249 >> v2:172.21.15.130:6804/3830191025 conn(0x7f0b1403d9d0 0x7f0b1403fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-16T07:50:12.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.413+0000 7f7d4273a640 1 --2- 172.21.15.130:0/407131219 >> v2:172.21.15.164:6806/3962925382 conn(0x7f7d2003d9c0 0x7f7d2003fe50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.413+0000 7f216253a640 1 --2- 172.21.15.130:0/3810228214 >> v2:172.21.15.164:6804/2835634474 conn(0x7f214003d9d0 0x7f214003fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-16T07:50:12.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.415+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== osd.3 v2:172.21.15.164:6806/3962925382 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f7d200404e0 con 0x7f7d2003d9c0 2024-04-16T07:50:12.443 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.416+0000 7f07f9e4a640 1 --2- 172.21.15.130:0/4190286811 >> v2:172.21.15.130:6802/3361768573 conn(0x7f07d803daa0 0x7f07d803ff30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.443 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.417+0000 7f0b32d9d640 1 --2- 172.21.15.130:0/194416249 >> v2:172.21.15.130:6804/3830191025 conn(0x7f0b1403d9d0 0x7f0b1403fe60 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.443 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.418+0000 7f216253a640 1 --2- 172.21.15.130:0/3810228214 >> v2:172.21.15.164:6804/2835634474 conn(0x7f214003d9d0 0x7f214003fe60 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.443 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.418+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 --> v2:172.21.15.164:6806/3962925382 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d24002c30 con 0x7f7d2003d9c0 2024-04-16T07:50:12.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.418+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== osd.0 v2:172.21.15.130:6802/3361768573 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f07d80405c0 con 0x7f07d803daa0 2024-04-16T07:50:12.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.419+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== osd.1 v2:172.21.15.130:6804/3830191025 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f0b140404f0 con 0x7f0b1403d9d0 2024-04-16T07:50:12.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== osd.2 v2:172.21.15.164:6804/2835634474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f21400404f0 con 0x7f214003d9d0 2024-04-16T07:50:12.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d327fc640 1 -- 172.21.15.130:0/407131219 <== osd.3 v2:172.21.15.164:6806/3962925382 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7d24002c30 con 0x7f7d2003d9c0 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 >> v2:172.21.15.164:6806/3962925382 conn(0x7f7d2003d9c0 msgr2=0x7f7d2003fe50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 --2- 172.21.15.130:0/407131219 >> v2:172.21.15.164:6806/3962925382 conn(0x7f7d2003d9c0 0x7f7d2003fe50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d200389c0 msgr2=0x7f7d2003aed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d200389c0 0x7f7d2003aed0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c09bae0 msgr2=0x7f7d3c11d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c09bae0 0x7f7d3c11d3d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 shutdown_connections 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 --2- 172.21.15.130:0/407131219 >> v2:172.21.15.164:6806/3962925382 conn(0x7f7d2003d9c0 0x7f7d2003fe50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.420+0000 7f7d4419d640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d200389c0 0x7f7d2003aed0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.421+0000 7f7d4419d640 1 --2- 172.21.15.130:0/407131219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d3c09bae0 0x7f7d3c11d3d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.421+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 >> 172.21.15.130:0/407131219 conn(0x7f7d3c0a3c20 msgr2=0x7f7d3c1bab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.421+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 shutdown_connections 2024-04-16T07:50:12.447 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.421+0000 7f7d4419d640 1 -- 172.21.15.130:0/407131219 wait complete. 2024-04-16T07:50:12.447 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.422+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 --> v2:172.21.15.130:6804/3830191025 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b3406e990 con 0x7f0b1403d9d0 2024-04-16T07:50:12.447 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.423+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 --> v2:172.21.15.130:6802/3361768573 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07f411dae0 con 0x7f07d803daa0 2024-04-16T07:50:12.447 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.424+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 --> v2:172.21.15.164:6804/2835634474 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f215c121150 con 0x7f214003d9d0 2024-04-16T07:50:12.448 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b2affd640 1 -- 172.21.15.130:0/194416249 <== osd.1 v2:172.21.15.130:6804/3830191025 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b3406e990 con 0x7f0b1403d9d0 2024-04-16T07:50:12.448 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 >> v2:172.21.15.130:6804/3830191025 conn(0x7f0b1403d9d0 msgr2=0x7f0b1403fe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.448 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/194416249 >> v2:172.21.15.130:6804/3830191025 conn(0x7f0b1403d9d0 0x7f0b1403fe60 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0b140389d0 msgr2=0x7f0b1403aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0b140389d0 0x7f0b1403aee0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b3409bae0 msgr2=0x7f0b341c9c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b3409bae0 0x7f0b341c9c30 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 shutdown_connections 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/194416249 >> v2:172.21.15.130:6804/3830191025 conn(0x7f0b1403d9d0 0x7f0b1403fe60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0b140389d0 0x7f0b1403aee0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 --2- 172.21.15.130:0/194416249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b3409bae0 0x7f0b341c9c30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 >> 172.21.15.130:0/194416249 conn(0x7f0b340a3c20 msgr2=0x7f0b34134fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 shutdown_connections 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.425+0000 7f0b38c6d640 1 -- 172.21.15.130:0/194416249 wait complete. 2024-04-16T07:50:12.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.426+0000 7f21527fc640 1 -- 172.21.15.130:0/3810228214 <== osd.2 v2:172.21.15.164:6804/2835634474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f215400a820 con 0x7f214003d9d0 2024-04-16T07:50:12.450 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 >> v2:172.21.15.164:6804/2835634474 conn(0x7f214003d9d0 msgr2=0x7f214003fe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.450 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 --2- 172.21.15.130:0/3810228214 >> v2:172.21.15.164:6804/2835634474 conn(0x7f214003d9d0 0x7f214003fe60 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.450 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f21400389d0 msgr2=0x7f214003aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.450 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f21400389d0 0x7f214003aee0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.450 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c09bae0 msgr2=0x7f215c11d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.450 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c09bae0 0x7f215c11d3d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.451 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f07e9ffb640 1 -- 172.21.15.130:0/4190286811 <== osd.0 v2:172.21.15.130:6802/3361768573 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f07ec00a820 con 0x7f07d803daa0 2024-04-16T07:50:12.451 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 shutdown_connections 2024-04-16T07:50:12.451 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 --2- 172.21.15.130:0/3810228214 >> v2:172.21.15.164:6804/2835634474 conn(0x7f214003d9d0 0x7f214003fe60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.451 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f21400389d0 0x7f214003aee0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.451 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 --2- 172.21.15.130:0/3810228214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f215c09bae0 0x7f215c11d3d0 crc :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.451 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 >> 172.21.15.130:0/3810228214 conn(0x7f215c0a3c20 msgr2=0x7f215c1bab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.452 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.427+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 shutdown_connections 2024-04-16T07:50:12.452 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f2163f9d640 1 -- 172.21.15.130:0/3810228214 wait complete. 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 >> v2:172.21.15.130:6802/3361768573 conn(0x7f07d803daa0 msgr2=0x7f07d803ff30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/4190286811 >> v2:172.21.15.130:6802/3361768573 conn(0x7f07d803daa0 0x7f07d803ff30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f07d8038a20 msgr2=0x7f07d803af30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f07d8038a20 0x7f07d803af30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 msgr2=0x7f07f411a290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f411a290 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 shutdown_connections 2024-04-16T07:50:12.453 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/4190286811 >> v2:172.21.15.130:6802/3361768573 conn(0x7f07d803daa0 0x7f07d803ff30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.454 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f07d8038a20 0x7f07d803af30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.454 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 --2- 172.21.15.130:0/4190286811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f40a36c0 0x7f07f411a290 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.454 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 >> 172.21.15.130:0/4190286811 conn(0x7f07f40a2ba0 msgr2=0x7f07f409d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.454 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 shutdown_connections 2024-04-16T07:50:12.454 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.428+0000 7f07fb8ad640 1 -- 172.21.15.130:0/4190286811 wait complete. 2024-04-16T07:50:12.454 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:12.455 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-16T07:50:12.456 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:12.456 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-16T07:50:12.456 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:12.457 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-16T07:50:12.457 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:12.458 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-16T07:50:12.620 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.615+0000 7f667439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.620 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.615+0000 7fa131cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.615+0000 7f0f11ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.615+0000 7fd5a4064640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.621 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.621+0000 7f0f11ead640 1 Processor -- start 2024-04-16T07:50:12.622 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.621+0000 7fa131cad640 1 Processor -- start 2024-04-16T07:50:12.622 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.621+0000 7f0f11ead640 1 -- start start 2024-04-16T07:50:12.622 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.621+0000 7fa131cad640 1 -- start start 2024-04-16T07:50:12.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.622+0000 7f667439d640 1 Processor -- start 2024-04-16T07:50:12.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.622+0000 7f667439d640 1 -- start start 2024-04-16T07:50:12.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.622+0000 7fd5a4064640 1 Processor -- start 2024-04-16T07:50:12.623 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.622+0000 7fd5a4064640 1 -- start start 2024-04-16T07:50:12.624 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f11ead640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f0f0c134150 0x7f0f0c136570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.624 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa131cad640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fa12c134be0 0x7fa12c137000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.624 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa131cad640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa12c09c6e0 con 0x7fa12c0a24f0 2024-04-16T07:50:12.624 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa131cad640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fa12c0735b0 con 0x7fa12c134be0 2024-04-16T07:50:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f11ead640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0c06f320 con 0x7f0f0c12de10 2024-04-16T07:50:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa12b7fe640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7fa12c0a24f0 0x7fa12c12eb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42558/0 (socket says 172.21.15.130:42558) 2024-04-16T07:50:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa12b7fe640 1 -- 172.21.15.130:0/4232021403 learned_addr learned my addr 172.21.15.130:0/4232021403 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f11ead640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f0f0c12d860 con 0x7f0f0c134150 2024-04-16T07:50:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f0affd640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f0f0c134150 0x7f0f0c136570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f0affd640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f0f0c134150 0x7f0f0c136570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58172/0 (socket says 172.21.15.130:58172) 2024-04-16T07:50:12.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f0affd640 1 -- 172.21.15.130:0/999518331 learned_addr learned my addr 172.21.15.130:0/999518331 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.626 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f667439d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f666c137ca0 0x7f666c13a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.626 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa12affd640 1 --2- 172.21.15.130:0/4232021403 >> v2:172.21.15.130:3300/0 conn(0x7fa12c134be0 0x7fa12c137000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.626 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f6671938640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f666c137ca0 0x7f666c13a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fd5a4064640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fd59c137cd0 0x7fd59c13a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f6671938640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f666c137ca0 0x7f666c13a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58176/0 (socket says 172.21.15.130:58176) 2024-04-16T07:50:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f6671938640 1 -- 172.21.15.130:0/1912029995 learned_addr learned my addr 172.21.15.130:0/1912029995 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f667439d640 1 -- 172.21.15.130:0/1912029995 --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c0a1c80 con 0x7f666c09af10 2024-04-16T07:50:12.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f667439d640 1 -- 172.21.15.130:0/1912029995 --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f666c0a10b0 con 0x7f666c137ca0 2024-04-16T07:50:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f0affd640 1 -- 172.21.15.130:0/999518331 >> v1:172.21.15.130:6789/0 conn(0x7f0f0c12de10 legacy=0x7f0f0c12e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7f0f0affd640 1 -- 172.21.15.130:0/999518331 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c09c9f0 con 0x7f0f0c134150 2024-04-16T07:50:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa12a7fc640 1 -- 172.21.15.130:0/4232021403 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226931335 0 0) 0x7fa12c09c6e0 con 0x7fa12c0a24f0 2024-04-16T07:50:12.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fd5a1e00640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7fd59c09b100 0x7fd59c09b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42580/0 (socket says 172.21.15.130:42580) 2024-04-16T07:50:12.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa12affd640 1 -- 172.21.15.130:0/4232021403 >> v1:172.21.15.130:6789/0 conn(0x7fa12c0a24f0 legacy=0x7fa12c12eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.624+0000 7fa12affd640 1 -- 172.21.15.130:0/4232021403 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c0739d0 con 0x7fa12c134be0 2024-04-16T07:50:12.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a15ff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fd59c137cd0 0x7fd59c13a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a15ff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7fd59c137cd0 0x7fd59c13a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58178/0 (socket says 172.21.15.130:58178) 2024-04-16T07:50:12.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f0f0affd640 1 --2- 172.21.15.130:0/999518331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c134150 0x7f0f0c136570 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f30f4fdb1853d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a4064640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd59c09aad0 con 0x7fd59c09b100 2024-04-16T07:50:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a4064640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7fd59c068980 con 0x7fd59c137cd0 2024-04-16T07:50:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a1e00640 1 -- 172.21.15.130:0/1162564306 learned_addr learned my addr 172.21.15.130:0/1162564306 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f6671938640 1 -- 172.21.15.130:0/1912029995 >> v1:172.21.15.130:6789/0 conn(0x7f666c09af10 legacy=0x7f666c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f6671938640 1 -- 172.21.15.130:0/1912029995 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c0a1970 con 0x7f666c137ca0 2024-04-16T07:50:12.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f6671938640 1 --2- 172.21.15.130:0/1912029995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c137ca0 0x7f666c13a0c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79603981538c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fa12affd640 1 --2- 172.21.15.130:0/4232021403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c134be0 0x7fa12c137000 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93606574edca53f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f0f0a7fc640 1 -- 172.21.15.130:0/999518331 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ef4009e10 con 0x7f0f0c134150 2024-04-16T07:50:12.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f0f0a7fc640 1 -- 172.21.15.130:0/999518331 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0ef400f470 con 0x7f0f0c134150 2024-04-16T07:50:12.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f0f0a7fc640 1 -- 172.21.15.130:0/999518331 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0ef400f710 con 0x7f0f0c134150 2024-04-16T07:50:12.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a15ff640 1 -- 172.21.15.130:0/1162564306 >> v1:172.21.15.130:6789/0 conn(0x7fd59c09b100 legacy=0x7fd59c09b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f6671137640 1 -- 172.21.15.130:0/1912029995 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f665c009e50 con 0x7f666c137ca0 2024-04-16T07:50:12.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a0dfe640 1 -- 172.21.15.130:0/1162564306 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4070144097 0 0) 0x7fd59c09aad0 con 0x7fd59c09b100 2024-04-16T07:50:12.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a15ff640 1 -- 172.21.15.130:0/1162564306 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c06ed00 con 0x7fd59c137cd0 2024-04-16T07:50:12.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f6671137640 1 -- 172.21.15.130:0/1912029995 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f665c00e230 con 0x7f666c137ca0 2024-04-16T07:50:12.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7f6671137640 1 -- 172.21.15.130:0/1912029995 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f665c00e4d0 con 0x7f666c137ca0 2024-04-16T07:50:12.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a15ff640 1 --2- 172.21.15.130:0/1162564306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c137cd0 0x7fd59c13a0f0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cb0eb1ab39da72 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fa12a7fc640 1 -- 172.21.15.130:0/4232021403 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa114009d90 con 0x7fa12c134be0 2024-04-16T07:50:12.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fa12a7fc640 1 -- 172.21.15.130:0/4232021403 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa114010440 con 0x7fa12c134be0 2024-04-16T07:50:12.634 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fa12a7fc640 1 -- 172.21.15.130:0/4232021403 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa1140106e0 con 0x7fa12c134be0 2024-04-16T07:50:12.634 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.625+0000 7fd5a0dfe640 1 -- 172.21.15.130:0/1162564306 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd58c009d90 con 0x7fd59c137cd0 2024-04-16T07:50:12.634 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7fd5a0dfe640 1 -- 172.21.15.130:0/1162564306 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd58c00e230 con 0x7fd59c137cd0 2024-04-16T07:50:12.634 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7fd5a0dfe640 1 -- 172.21.15.130:0/1162564306 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd58c00e4d0 con 0x7fd59c137cd0 2024-04-16T07:50:12.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 1 -- 172.21.15.130:0/999518331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c134150 msgr2=0x7f0f0c136570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 1 --2- 172.21.15.130:0/999518331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c134150 0x7f0f0c136570 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 1 -- 172.21.15.130:0/999518331 shutdown_connections 2024-04-16T07:50:12.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 1 --2- 172.21.15.130:0/999518331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c134150 0x7f0f0c136570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 1 -- 172.21.15.130:0/999518331 >> 172.21.15.130:0/999518331 conn(0x7f0f0c0a09c0 msgr2=0x7f0f0c0a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 1 -- 172.21.15.130:0/999518331 shutdown_connections 2024-04-16T07:50:12.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 1 -- 172.21.15.130:0/999518331 wait complete. 2024-04-16T07:50:12.636 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f667439d640 1 -- 172.21.15.130:0/1912029995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c137ca0 msgr2=0x7f666c13a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.636 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f667439d640 1 --2- 172.21.15.130:0/1912029995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c137ca0 0x7f666c13a0c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.636 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.626+0000 7f0f11ead640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.637 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7f667439d640 1 -- 172.21.15.130:0/1912029995 shutdown_connections 2024-04-16T07:50:12.637 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7f667439d640 1 --2- 172.21.15.130:0/1912029995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c137ca0 0x7f666c13a0c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.637 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7f667439d640 1 -- 172.21.15.130:0/1912029995 >> 172.21.15.130:0/1912029995 conn(0x7f666c0a3c90 msgr2=0x7f666c0a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.637 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7f667439d640 1 -- 172.21.15.130:0/1912029995 shutdown_connections 2024-04-16T07:50:12.637 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7f667439d640 1 -- 172.21.15.130:0/1912029995 wait complete. 2024-04-16T07:50:12.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 1 -- 172.21.15.130:0/4232021403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c134be0 msgr2=0x7fa12c137000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 1 --2- 172.21.15.130:0/4232021403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c134be0 0x7fa12c137000 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 1 -- 172.21.15.130:0/4232021403 shutdown_connections 2024-04-16T07:50:12.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 1 --2- 172.21.15.130:0/4232021403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c134be0 0x7fa12c137000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 1 -- 172.21.15.130:0/4232021403 >> 172.21.15.130:0/4232021403 conn(0x7fa12c0a09c0 msgr2=0x7fa12c0a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7f667439d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fd5a4064640 1 -- 172.21.15.130:0/1162564306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c137cd0 msgr2=0x7fd59c13a0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fd5a4064640 1 --2- 172.21.15.130:0/1162564306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c137cd0 0x7fd59c13a0f0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 1 -- 172.21.15.130:0/4232021403 shutdown_connections 2024-04-16T07:50:12.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 1 -- 172.21.15.130:0/4232021403 wait complete. 2024-04-16T07:50:12.640 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fd5a4064640 1 -- 172.21.15.130:0/1162564306 shutdown_connections 2024-04-16T07:50:12.640 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fd5a4064640 1 --2- 172.21.15.130:0/1162564306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c137cd0 0x7fd59c13a0f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.640 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fd5a4064640 1 -- 172.21.15.130:0/1162564306 >> 172.21.15.130:0/1162564306 conn(0x7fd59c0a09c0 msgr2=0x7fd59c0a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.640 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fa131cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.641 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fd5a4064640 1 -- 172.21.15.130:0/1162564306 shutdown_connections 2024-04-16T07:50:12.641 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.627+0000 7fd5a4064640 1 -- 172.21.15.130:0/1162564306 wait complete. 2024-04-16T07:50:12.641 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.628+0000 7fd5a4064640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:12.641 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.628+0000 7f0f11ead640 1 Processor -- start 2024-04-16T07:50:12.641 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.628+0000 7f667439d640 1 Processor -- start 2024-04-16T07:50:12.642 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.628+0000 7f667439d640 1 -- start start 2024-04-16T07:50:12.642 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.629+0000 7fa131cad640 1 Processor -- start 2024-04-16T07:50:12.642 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.629+0000 7f0f11ead640 1 -- start start 2024-04-16T07:50:12.642 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.629+0000 7fa131cad640 1 -- start start 2024-04-16T07:50:12.643 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7f667439d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c09af10 0x7f666c11f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.643 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7f667439d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f666c11f480 con 0x7f666c09af10 2024-04-16T07:50:12.643 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7f6672139640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c09af10 0x7f666c11f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.643 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7fa131cad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c0a24f0 0x7fa12c11a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.643 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7f6672139640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c09af10 0x7f666c11f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58184/0 (socket says 172.21.15.130:58184) 2024-04-16T07:50:12.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7fa131cad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c11b740 con 0x7fa12c0a24f0 2024-04-16T07:50:12.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7f6672139640 1 -- 172.21.15.130:0/277681493 learned_addr learned my addr 172.21.15.130:0/277681493 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c0a24f0 0x7fa12c11a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7fd5a4064640 1 Processor -- start 2024-04-16T07:50:12.645 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c0a24f0 0x7fa12c11a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58196/0 (socket says 172.21.15.130:58196) 2024-04-16T07:50:12.645 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7fa12b7fe640 1 -- 172.21.15.130:0/1466361310 learned_addr learned my addr 172.21.15.130:0/1466361310 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.645 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7fd5a4064640 1 -- start start 2024-04-16T07:50:12.646 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7f6672139640 1 -- 172.21.15.130:0/277681493 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c121990 con 0x7f666c09af10 2024-04-16T07:50:12.646 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.630+0000 7f6672139640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c09af10 0x7f666c11f840 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.646 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f0f11ead640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c12de10 0x7f0f0c072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.646 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f0f11ead640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c072500 con 0x7f0f0c12de10 2024-04-16T07:50:12.646 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7fa12b7fe640 1 -- 172.21.15.130:0/1466361310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c11a140 con 0x7fa12c0a24f0 2024-04-16T07:50:12.647 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f0f0b7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c12de10 0x7f0f0c072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.647 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f6662ffd640 1 -- 172.21.15.130:0/277681493 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f666800c870 con 0x7f666c09af10 2024-04-16T07:50:12.647 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f6662ffd640 1 -- 172.21.15.130:0/277681493 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f666800c9d0 con 0x7f666c09af10 2024-04-16T07:50:12.647 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f6662ffd640 1 -- 172.21.15.130:0/277681493 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f666800cc70 con 0x7f666c09af10 2024-04-16T07:50:12.647 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c121250 con 0x7f666c09af10 2024-04-16T07:50:12.647 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c122df0 con 0x7f666c09af10 2024-04-16T07:50:12.648 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f0f0b7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c12de10 0x7f0f0c072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58212/0 (socket says 172.21.15.130:58212) 2024-04-16T07:50:12.648 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f0f0b7fe640 1 -- 172.21.15.130:0/3773279658 learned_addr learned my addr 172.21.15.130:0/3773279658 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.648 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7fa12b7fe640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c0a24f0 0x7fa12c11a5c0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.649 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7f0f0b7fe640 1 -- 172.21.15.130:0/3773279658 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c11b850 con 0x7f0f0c12de10 2024-04-16T07:50:12.649 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7fa123fff640 1 -- 172.21.15.130:0/1466361310 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa124004bc0 con 0x7fa12c0a24f0 2024-04-16T07:50:12.649 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7fa123fff640 1 -- 172.21.15.130:0/1466361310 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa124004d20 con 0x7fa12c0a24f0 2024-04-16T07:50:12.649 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.631+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c11ce80 con 0x7fa12c0a24f0 2024-04-16T07:50:12.649 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7f0f0b7fe640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c12de10 0x7f0f0c072b60 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7fa123fff640 1 -- 172.21.15.130:0/1466361310 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa124004bc0 con 0x7fa12c0a24f0 2024-04-16T07:50:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c11d740 con 0x7fa12c0a24f0 2024-04-16T07:50:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7fd5a4064640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c09b100 0x7fd59c11f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.650 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7fd5a4064640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c120360 con 0x7fd59c09b100 2024-04-16T07:50:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c09b100 0x7fd59c11f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7fd5a1e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c09b100 0x7fd59c11f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58218/0 (socket says 172.21.15.130:58218) 2024-04-16T07:50:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7fd5a1e00640 1 -- 172.21.15.130:0/3165269067 learned_addr learned my addr 172.21.15.130:0/3165269067 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7f6662ffd640 1 -- 172.21.15.130:0/277681493 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f6668011310 con 0x7f666c09af10 2024-04-16T07:50:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7f0f03fff640 1 -- 172.21.15.130:0/3773279658 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0efc00cbf0 con 0x7f0f0c12de10 2024-04-16T07:50:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7f0f03fff640 1 -- 172.21.15.130:0/3773279658 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0efc00cd50 con 0x7f0f0c12de10 2024-04-16T07:50:12.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.632+0000 7f0f03fff640 1 -- 172.21.15.130:0/3773279658 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0efc00f230 con 0x7f0f0c12de10 2024-04-16T07:50:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7f6662ffd640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f6650038980 0x7f665003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7f6662ffd640 1 -- 172.21.15.130:0/277681493 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f66680488e0 con 0x7f666c09af10 2024-04-16T07:50:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c122690 con 0x7f666c09af10 2024-04-16T07:50:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c073db0 con 0x7f0f0c12de10 2024-04-16T07:50:12.652 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c072160 con 0x7f0f0c12de10 2024-04-16T07:50:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7fd5a1e00640 1 -- 172.21.15.130:0/3165269067 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c1226d0 con 0x7fd59c09b100 2024-04-16T07:50:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12c11d070 con 0x7fa12c0a24f0 2024-04-16T07:50:12.653 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7fd5a1e00640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c09b100 0x7fd59c11f060 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7fa123fff640 1 -- 172.21.15.130:0/1466361310 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fa124007120 con 0x7fa12c0a24f0 2024-04-16T07:50:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7fa123fff640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fa10c038970 0x7fa10c03ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7fa123fff640 1 -- 172.21.15.130:0/1466361310 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fa1240488b0 con 0x7fa12c0a24f0 2024-04-16T07:50:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.633+0000 7f6671938640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f6650038980 0x7f665003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.634+0000 7fa12affd640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fa10c038970 0x7fa10c03ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.634+0000 7f0f03fff640 1 -- 172.21.15.130:0/3773279658 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f0efc00f390 con 0x7f0f0c12de10 2024-04-16T07:50:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.634+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0c063930 con 0x7f0f0c12de10 2024-04-16T07:50:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.634+0000 7f0f03fff640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0eec0389d0 0x7f0eec03aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.634+0000 7f6671938640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f6650038980 0x7f665003ae90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.656 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.634+0000 7fa12affd640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fa10c038970 0x7fa10c03ae80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.656 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.634+0000 7f0f0affd640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0eec0389d0 0x7f0eec03aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.656 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.635+0000 7f0f0affd640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0eec0389d0 0x7f0eec03aee0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.656 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.639+0000 7f0f03fff640 1 -- 172.21.15.130:0/3773279658 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f0efc00abe0 con 0x7f0f0c12de10 2024-04-16T07:50:12.656 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.639+0000 7f6662ffd640 1 -- 172.21.15.130:0/277681493 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f666800adc0 con 0x7f666c09af10 2024-04-16T07:50:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.639+0000 7fd5927fc640 1 -- 172.21.15.130:0/3165269067 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd598004b80 con 0x7fd59c09b100 2024-04-16T07:50:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.639+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c120c80 con 0x7fd59c09b100 2024-04-16T07:50:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.639+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c073db0 con 0x7fd59c09b100 2024-04-16T07:50:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.640+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59c123170 con 0x7fd59c09b100 2024-04-16T07:50:12.657 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.644+0000 7fa123fff640 1 -- 172.21.15.130:0/1466361310 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7fa1240062c0 con 0x7fa12c0a24f0 2024-04-16T07:50:12.658 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.649+0000 7fd5927fc640 1 -- 172.21.15.130:0/3165269067 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd598004ce0 con 0x7fd59c09b100 2024-04-16T07:50:12.658 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.649+0000 7fd5927fc640 1 -- 172.21.15.130:0/3165269067 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd598007120 con 0x7fd59c09b100 2024-04-16T07:50:12.658 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.649+0000 7f0f03fff640 1 -- 172.21.15.130:0/3773279658 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f0efc00adf0 con 0x7f0f0c12de10 2024-04-16T07:50:12.659 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.650+0000 7fd5927fc640 1 -- 172.21.15.130:0/3165269067 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fd598007280 con 0x7fd59c09b100 2024-04-16T07:50:12.659 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.651+0000 7fd5927fc640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fd580038980 0x7fd58003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:12.659 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.651+0000 7fd5927fc640 1 -- 172.21.15.130:0/3165269067 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fd59800d2f0 con 0x7fd59c09b100 2024-04-16T07:50:12.659 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.651+0000 7fd5a15ff640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fd580038980 0x7fd58003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:12.659 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.652+0000 7fd5a15ff640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fd580038980 0x7fd58003ae90 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:12.659 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.655+0000 7fd5927fc640 1 -- 172.21.15.130:0/3165269067 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7fd59800ce10 con 0x7fd59c09b100 2024-04-16T07:50:12.830 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.830+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa12c134390 con 0x7fa12c0a24f0 2024-04-16T07:50:12.830 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.831+0000 7fa123fff640 1 -- 172.21.15.130:0/1466361310 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7fa12400ad00 con 0x7fa12c0a24f0 2024-04-16T07:50:12.830 INFO:teuthology.orchestra.run.smithi130.stdout:38654705669 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fa10c038970 msgr2=0x7fa10c03ae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fa10c038970 0x7fa10c03ae80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c0a24f0 msgr2=0x7fa12c11a5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c0a24f0 0x7fa12c11a5c0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 shutdown_connections 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fa10c038970 0x7fa10c03ae80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 --2- 172.21.15.130:0/1466361310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa12c0a24f0 0x7fa12c11a5c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 >> 172.21.15.130:0/1466361310 conn(0x7fa12c0a09c0 msgr2=0x7fa12c0a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 shutdown_connections 2024-04-16T07:50:12.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.834+0000 7fa131cad640 1 -- 172.21.15.130:0/1466361310 wait complete. 2024-04-16T07:50:12.841 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.842+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f666c121770 con 0x7f666c09af10 2024-04-16T07:50:12.842 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.842+0000 7f6662ffd640 1 -- 172.21.15.130:0/277681493 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f666800a9b0 con 0x7f666c09af10 2024-04-16T07:50:12.842 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:12.844 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.844+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f6650038980 msgr2=0x7f665003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.844+0000 7f667439d640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f6650038980 0x7f665003ae90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.844+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c09af10 msgr2=0x7f666c11f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.844+0000 7f667439d640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c09af10 0x7f666c11f840 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.845+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 shutdown_connections 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.845+0000 7f667439d640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f6650038980 0x7f665003ae90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.845+0000 7f667439d640 1 --2- 172.21.15.130:0/277681493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f666c09af10 0x7f666c11f840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.845+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 >> 172.21.15.130:0/277681493 conn(0x7f666c0a3c90 msgr2=0x7f666c1347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.845+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 shutdown_connections 2024-04-16T07:50:12.845 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.845+0000 7f667439d640 1 -- 172.21.15.130:0/277681493 wait complete. 2024-04-16T07:50:12.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2024-04-16T07:50:12.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.849+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd59c063930 con 0x7fd59c09b100 2024-04-16T07:50:12.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.849+0000 7fd5927fc640 1 -- 172.21.15.130:0/3165269067 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7fd59800e1c0 con 0x7fd59c09b100 2024-04-16T07:50:12.849 INFO:teuthology.orchestra.run.smithi130.stdout:38654705669 2024-04-16T07:50:12.851 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fd580038980 msgr2=0x7fd58003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.851 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fd580038980 0x7fd58003ae90 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.851 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c09b100 msgr2=0x7fd59c11f060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.851 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c09b100 0x7fd59c11f060 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 shutdown_connections 2024-04-16T07:50:12.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fd580038980 0x7fd58003ae90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 --2- 172.21.15.130:0/3165269067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd59c09b100 0x7fd59c11f060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.851+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 >> 172.21.15.130:0/3165269067 conn(0x7fd59c0a09c0 msgr2=0x7fd59c134890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.852+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 shutdown_connections 2024-04-16T07:50:12.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.852+0000 7fd5a4064640 1 -- 172.21.15.130:0/3165269067 wait complete. 2024-04-16T07:50:12.852 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.853+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0f0c09ced0 con 0x7f0f0c12de10 2024-04-16T07:50:12.853 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.853+0000 7f0f03fff640 1 -- 172.21.15.130:0/3773279658 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f0efc047e40 con 0x7f0f0c12de10 2024-04-16T07:50:12.853 INFO:teuthology.orchestra.run.smithi130.stdout:38654705669 2024-04-16T07:50:12.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0eec0389d0 msgr2=0x7f0eec03aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0eec0389d0 0x7f0eec03aee0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c12de10 msgr2=0x7f0f0c072b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c12de10 0x7f0f0c072b60 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 shutdown_connections 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0eec0389d0 0x7f0eec03aee0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 --2- 172.21.15.130:0/3773279658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f0c12de10 0x7f0f0c072b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.855+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 >> 172.21.15.130:0/3773279658 conn(0x7f0f0c0a09c0 msgr2=0x7f0f0c12bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.856+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 shutdown_connections 2024-04-16T07:50:12.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:12.856+0000 7f0f11ead640 1 -- 172.21.15.130:0/3773279658 wait complete. 2024-04-16T07:50:12.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.3 2024-04-16T07:50:12.857 DEBUG:teuthology.parallel:result is None 2024-04-16T07:50:12.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-04-16T07:50:12.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2024-04-16T07:50:13.849 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-16T07:50:13.865 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-16T07:50:13.868 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-16T07:50:13.975 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.968+0000 7f2a45864640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:13.976 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.976+0000 7f2a45864640 1 Processor -- start 2024-04-16T07:50:13.976 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.976+0000 7f2a45864640 1 -- start start 2024-04-16T07:50:13.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.978+0000 7f2a45864640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f2a40137d40 0x7f2a4013a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:13.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.978+0000 7f2a45864640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a400a2b30 con 0x7f2a400a2010 2024-04-16T07:50:13.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.978+0000 7f2a45864640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f2a400a12d0 con 0x7f2a40137d40 2024-04-16T07:50:13.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.978+0000 7f2a3e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f2a40137d40 0x7f2a4013a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:13.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.978+0000 7f2a3e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f2a40137d40 0x7f2a4013a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58222/0 (socket says 172.21.15.130:58222) 2024-04-16T07:50:13.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.978+0000 7f2a3effd640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f2a400a2010 0x7f2a400a2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42594/0 (socket says 172.21.15.130:42594) 2024-04-16T07:50:13.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.978+0000 7f2a3e7fc640 1 -- 172.21.15.130:0/2355248241 learned_addr learned my addr 172.21.15.130:0/2355248241 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.979+0000 7f2a3e7fc640 1 -- 172.21.15.130:0/2355248241 >> v1:172.21.15.130:6789/0 conn(0x7f2a400a2010 legacy=0x7f2a400a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.979+0000 7f2a3e7fc640 1 -- 172.21.15.130:0/2355248241 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a400a1b30 con 0x7f2a40137d40 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.979+0000 7f2a3e7fc640 1 --2- 172.21.15.130:0/2355248241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a40137d40 0x7f2a4013a160 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29fa333b2ac41fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.979+0000 7f2a3dffb640 1 -- 172.21.15.130:0/2355248241 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a2800e040 con 0x7f2a40137d40 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.979+0000 7f2a3dffb640 1 -- 172.21.15.130:0/2355248241 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2a280091e0 con 0x7f2a40137d40 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.979+0000 7f2a3dffb640 1 -- 172.21.15.130:0/2355248241 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a28007730 con 0x7f2a40137d40 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 1 -- 172.21.15.130:0/2355248241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a40137d40 msgr2=0x7f2a4013a160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:13.980 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 1 --2- 172.21.15.130:0/2355248241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a40137d40 0x7f2a4013a160 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 1 -- 172.21.15.130:0/2355248241 shutdown_connections 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 1 --2- 172.21.15.130:0/2355248241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a40137d40 0x7f2a4013a160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 1 -- 172.21.15.130:0/2355248241 >> 172.21.15.130:0/2355248241 conn(0x7f2a400a3c20 msgr2=0x7f2a400a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 1 -- 172.21.15.130:0/2355248241 shutdown_connections 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 1 -- 172.21.15.130:0/2355248241 wait complete. 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.980+0000 7f2a45864640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.982+0000 7f2a45864640 1 Processor -- start 2024-04-16T07:50:13.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.982+0000 7f2a45864640 1 -- start start 2024-04-16T07:50:13.983 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.983+0000 7f2a45864640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a400a2010 0x7f2a4011ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:13.983 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.983+0000 7f2a45864640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40122530 con 0x7f2a400a2010 2024-04-16T07:50:13.983 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.983+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a400a2010 0x7f2a4011ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:13.983 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.983+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a400a2010 0x7f2a4011ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58228/0 (socket says 172.21.15.130:58228) 2024-04-16T07:50:13.983 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.983+0000 7f2a3effd640 1 -- 172.21.15.130:0/1553219739 learned_addr learned my addr 172.21.15.130:0/1553219739 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:13.984 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.984+0000 7f2a3effd640 1 -- 172.21.15.130:0/1553219739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40132360 con 0x7f2a400a2010 2024-04-16T07:50:13.984 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.984+0000 7f2a3effd640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a400a2010 0x7f2a4011ef60 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:13.985 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.984+0000 7f2a37fff640 1 -- 172.21.15.130:0/1553219739 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a300059a0 con 0x7f2a400a2010 2024-04-16T07:50:13.985 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.984+0000 7f2a37fff640 1 -- 172.21.15.130:0/1553219739 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2a30005b00 con 0x7f2a400a2010 2024-04-16T07:50:13.985 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.984+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a40121330 con 0x7f2a400a2010 2024-04-16T07:50:13.985 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.984+0000 7f2a37fff640 1 -- 172.21.15.130:0/1553219739 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a30005da0 con 0x7f2a400a2010 2024-04-16T07:50:13.985 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.984+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a40122f40 con 0x7f2a400a2010 2024-04-16T07:50:13.985 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.985+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a401216e0 con 0x7f2a400a2010 2024-04-16T07:50:13.985 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.985+0000 7f2a37fff640 1 -- 172.21.15.130:0/1553219739 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f2a300070b0 con 0x7f2a400a2010 2024-04-16T07:50:13.986 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.986+0000 7f2a37fff640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f2a24038980 0x7f2a2403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:13.986 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.986+0000 7f2a37fff640 1 -- 172.21.15.130:0/1553219739 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f2a30048200 con 0x7f2a400a2010 2024-04-16T07:50:13.986 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.986+0000 7f2a3e7fc640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f2a24038980 0x7f2a2403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:13.986 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.987+0000 7f2a3e7fc640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f2a24038980 0x7f2a2403ae90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:13.995 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:13.996+0000 7f2a37fff640 1 -- 172.21.15.130:0/1553219739 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f2a3000d330 con 0x7f2a400a2010 2024-04-16T07:50:14.013 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.009+0000 7efc1ed5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:14.015 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.015+0000 7efc1ed5d640 1 Processor -- start 2024-04-16T07:50:14.015 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.015+0000 7efc1ed5d640 1 -- start start 2024-04-16T07:50:14.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc1ed5d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7efc18137ca0 0x7efc1813a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:14.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc1ed5d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc180a1c80 con 0x7efc1809af10 2024-04-16T07:50:14.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc1ed5d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7efc180a10b0 con 0x7efc18137ca0 2024-04-16T07:50:14.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc0ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7efc18137ca0 0x7efc1813a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:14.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc1caf9640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7efc1809af10 0x7efc180a22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42606/0 (socket says 172.21.15.130:42606) 2024-04-16T07:50:14.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc0ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7efc18137ca0 0x7efc1813a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58232/0 (socket says 172.21.15.130:58232) 2024-04-16T07:50:14.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc1caf9640 1 -- 172.21.15.130:0/934711003 learned_addr learned my addr 172.21.15.130:0/934711003 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:14.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc0ffff640 1 -- 172.21.15.130:0/934711003 >> v1:172.21.15.130:6789/0 conn(0x7efc1809af10 legacy=0x7efc180a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc0f7fe640 1 -- 172.21.15.130:0/934711003 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200409218 0 0) 0x7efc180a1c80 con 0x7efc1809af10 2024-04-16T07:50:14.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc0ffff640 1 -- 172.21.15.130:0/934711003 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc180a1970 con 0x7efc18137ca0 2024-04-16T07:50:14.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc0ffff640 1 --2- 172.21.15.130:0/934711003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc18137ca0 0x7efc1813a0c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4851e4d6cfc6f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:14.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.017+0000 7efc0f7fe640 1 -- 172.21.15.130:0/934711003 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc08009d90 con 0x7efc18137ca0 2024-04-16T07:50:14.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc0f7fe640 1 -- 172.21.15.130:0/934711003 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc0800e230 con 0x7efc18137ca0 2024-04-16T07:50:14.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc0f7fe640 1 -- 172.21.15.130:0/934711003 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc0800e4d0 con 0x7efc18137ca0 2024-04-16T07:50:14.019 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc1ed5d640 1 -- 172.21.15.130:0/934711003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc18137ca0 msgr2=0x7efc1813a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc1ed5d640 1 --2- 172.21.15.130:0/934711003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc18137ca0 0x7efc1813a0c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc1ed5d640 1 -- 172.21.15.130:0/934711003 shutdown_connections 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc1ed5d640 1 --2- 172.21.15.130:0/934711003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc18137ca0 0x7efc1813a0c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc1ed5d640 1 -- 172.21.15.130:0/934711003 >> 172.21.15.130:0/934711003 conn(0x7efc180a3c90 msgr2=0x7efc180a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc1ed5d640 1 -- 172.21.15.130:0/934711003 shutdown_connections 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.018+0000 7efc1ed5d640 1 -- 172.21.15.130:0/934711003 wait complete. 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.019+0000 7efc1ed5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.020+0000 7efc1ed5d640 1 Processor -- start 2024-04-16T07:50:14.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.020+0000 7efc1ed5d640 1 -- start start 2024-04-16T07:50:14.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.021+0000 7efc1ed5d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1809af10 0x7efc1811f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.021+0000 7efc1ed5d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efc181209f0 con 0x7efc1809af10 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.021+0000 7efc1caf9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1809af10 0x7efc1811f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.021+0000 7efc1caf9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1809af10 0x7efc1811f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58242/0 (socket says 172.21.15.130:58242) 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.021+0000 7efc1caf9640 1 -- 172.21.15.130:0/393486930 learned_addr learned my addr 172.21.15.130:0/393486930 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.022+0000 7efc1caf9640 1 -- 172.21.15.130:0/393486930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1811ee60 con 0x7efc1809af10 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.022+0000 7efc1caf9640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1809af10 0x7efc1811f960 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.022+0000 7efc0d7fa640 1 -- 172.21.15.130:0/393486930 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc14004b80 con 0x7efc1809af10 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.022+0000 7efc0d7fa640 1 -- 172.21.15.130:0/393486930 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc14004ce0 con 0x7efc1809af10 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.022+0000 7efc0d7fa640 1 -- 172.21.15.130:0/393486930 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc14007120 con 0x7efc1809af10 2024-04-16T07:50:14.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.022+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc18122e30 con 0x7efc1809af10 2024-04-16T07:50:14.023 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.022+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1811f090 con 0x7efc1809af10 2024-04-16T07:50:14.023 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.023+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc18063930 con 0x7efc1809af10 2024-04-16T07:50:14.023 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.023+0000 7efc0d7fa640 1 -- 172.21.15.130:0/393486930 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7efc140072d0 con 0x7efc1809af10 2024-04-16T07:50:14.023 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.024+0000 7efc0d7fa640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc04038980 0x7efc0403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:14.023 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.024+0000 7efc0d7fa640 1 -- 172.21.15.130:0/393486930 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7efc14048310 con 0x7efc1809af10 2024-04-16T07:50:14.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.024+0000 7efc0ffff640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc04038980 0x7efc0403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:14.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.025+0000 7efc0ffff640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc04038980 0x7efc0403ae90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:14.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.019+0000 7f76c14ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:14.026 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.026+0000 7f76c14ad640 1 Processor -- start 2024-04-16T07:50:14.026 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.026+0000 7f76c14ad640 1 -- start start 2024-04-16T07:50:14.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.027+0000 7f76c14ad640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f76bc09af10 0x7f76bc0a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:14.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.027+0000 7f76c14ad640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76bc0a1c80 con 0x7f76bc0a2810 2024-04-16T07:50:14.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.027+0000 7f76c14ad640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f76bc0a10b0 con 0x7f76bc09af10 2024-04-16T07:50:14.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.027+0000 7f76baffd640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f76bc09af10 0x7f76bc0a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:14.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.027+0000 7f76b3fff640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f76bc0a2810 0x7f76bc134930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42608/0 (socket says 172.21.15.130:42608) 2024-04-16T07:50:14.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.027+0000 7f76b3fff640 1 -- 172.21.15.130:0/325100607 learned_addr learned my addr 172.21.15.130:0/325100607 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:14.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.028+0000 7f76baffd640 1 -- 172.21.15.130:0/325100607 >> v1:172.21.15.130:6789/0 conn(0x7f76bc0a2810 legacy=0x7f76bc134930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.028+0000 7f76baffd640 1 -- 172.21.15.130:0/325100607 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc0a1970 con 0x7f76bc09af10 2024-04-16T07:50:14.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.028+0000 7f76baffd640 1 --2- 172.21.15.130:0/325100607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc0a22d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb418bbaab75e253 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:14.029 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.029+0000 7f76ba7fc640 1 -- 172.21.15.130:0/325100607 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f76ac009d90 con 0x7f76bc09af10 2024-04-16T07:50:14.029 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.029+0000 7f76ba7fc640 1 -- 172.21.15.130:0/325100607 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f76ac00e230 con 0x7f76bc09af10 2024-04-16T07:50:14.029 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.029+0000 7f76ba7fc640 1 -- 172.21.15.130:0/325100607 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f76ac00e4d0 con 0x7f76bc09af10 2024-04-16T07:50:14.029 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.029+0000 7efc0d7fa640 1 -- 172.21.15.130:0/393486930 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7efc1400a9e0 con 0x7efc1809af10 2024-04-16T07:50:14.030 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 1 -- 172.21.15.130:0/325100607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 msgr2=0x7f76bc0a22d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.030 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 1 --2- 172.21.15.130:0/325100607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc0a22d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.030 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 1 -- 172.21.15.130:0/325100607 shutdown_connections 2024-04-16T07:50:14.030 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 1 --2- 172.21.15.130:0/325100607 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc0a22d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.031 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 1 -- 172.21.15.130:0/325100607 >> 172.21.15.130:0/325100607 conn(0x7f76bc0a3c90 msgr2=0x7f76bc0a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:14.031 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 1 -- 172.21.15.130:0/325100607 shutdown_connections 2024-04-16T07:50:14.031 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 1 -- 172.21.15.130:0/325100607 wait complete. 2024-04-16T07:50:14.031 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.030+0000 7f76c14ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:14.031 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.032+0000 7f76c14ad640 1 Processor -- start 2024-04-16T07:50:14.031 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.032+0000 7f76c14ad640 1 -- start start 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.033+0000 7f76c14ad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc11df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.033+0000 7f76c14ad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc122070 con 0x7f76bc09af10 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.033+0000 7f76baffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc11df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.033+0000 7f76baffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc11df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58264/0 (socket says 172.21.15.130:58264) 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.033+0000 7f76baffd640 1 -- 172.21.15.130:0/734402159 learned_addr learned my addr 172.21.15.130:0/734402159 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.033+0000 7f76baffd640 1 -- 172.21.15.130:0/734402159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc1320e0 con 0x7f76bc09af10 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.033+0000 7f76baffd640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc11df80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:14.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.034+0000 7f76b37fe640 1 -- 172.21.15.130:0/734402159 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f76ac014620 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.034+0000 7f76b37fe640 1 -- 172.21.15.130:0/734402159 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f76ac014780 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.034+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc1203f0 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.034+0000 7f76b37fe640 1 -- 172.21.15.130:0/734402159 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f76ac014a20 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.034+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc121550 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.035+0000 7f76b37fe640 1 -- 172.21.15.130:0/734402159 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f76ac014bd0 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.035+0000 7f76b37fe640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f769c038980 0x7f769c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.035+0000 7f76b37fe640 1 -- 172.21.15.130:0/734402159 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f76ac019040 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.035+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76bc121e20 con 0x7f76bc09af10 2024-04-16T07:50:14.035 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.035+0000 7f76b3fff640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f769c038980 0x7f769c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:14.036 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.036+0000 7f76b3fff640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f769c038980 0x7f769c03ae90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:14.046 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.046+0000 7f76b37fe640 1 -- 172.21.15.130:0/734402159 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f76bc121e20 con 0x7f76bc09af10 2024-04-16T07:50:14.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.196+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2a40063930 con 0x7f2a400a2010 2024-04-16T07:50:14.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.197+0000 7f2a37fff640 1 -- 172.21.15.130:0/1553219739 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f2a3000d150 con 0x7f2a400a2010 2024-04-16T07:50:14.196 INFO:teuthology.orchestra.run.smithi130.stdout:38654705669 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f2a24038980 msgr2=0x7f2a2403ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f2a24038980 0x7f2a2403ae90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a400a2010 msgr2=0x7f2a4011ef60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a400a2010 0x7f2a4011ef60 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 shutdown_connections 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f2a24038980 0x7f2a2403ae90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 --2- 172.21.15.130:0/1553219739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2a400a2010 0x7f2a4011ef60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 >> 172.21.15.130:0/1553219739 conn(0x7f2a400a3c20 msgr2=0x7f2a40134820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 shutdown_connections 2024-04-16T07:50:14.199 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.199+0000 7f2a45864640 1 -- 172.21.15.130:0/1553219739 wait complete. 2024-04-16T07:50:14.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2024-04-16T07:50:14.232 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.232+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efc180008d0 con 0x7efc1809af10 2024-04-16T07:50:14.233 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.233+0000 7efc0d7fa640 1 -- 172.21.15.130:0/393486930 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7efc1400a5d0 con 0x7efc1809af10 2024-04-16T07:50:14.233 INFO:teuthology.orchestra.run.smithi130.stdout:38654705669 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc04038980 msgr2=0x7efc0403ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc04038980 0x7efc0403ae90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1809af10 msgr2=0x7efc1811f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1809af10 0x7efc1811f960 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 shutdown_connections 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7efc04038980 0x7efc0403ae90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 --2- 172.21.15.130:0/393486930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1809af10 0x7efc1811f960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 >> 172.21.15.130:0/393486930 conn(0x7efc180a3c90 msgr2=0x7efc181347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 shutdown_connections 2024-04-16T07:50:14.237 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.237+0000 7efc1ed5d640 1 -- 172.21.15.130:0/393486930 wait complete. 2024-04-16T07:50:14.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-04-16T07:50:14.254 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.255+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f76bc121e20 con 0x7f76bc09af10 2024-04-16T07:50:14.255 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.255+0000 7f76b37fe640 1 -- 172.21.15.130:0/734402159 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f76bc121e20 con 0x7f76bc09af10 2024-04-16T07:50:14.255 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:14.257 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f769c038980 msgr2=0x7f769c03ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.257 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f769c038980 0x7f769c03ae90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.257 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 msgr2=0x7f76bc11df80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc11df80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 shutdown_connections 2024-04-16T07:50:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f769c038980 0x7f769c03ae90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 --2- 172.21.15.130:0/734402159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76bc09af10 0x7f76bc11df80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 >> 172.21.15.130:0/734402159 conn(0x7f76bc0a3c90 msgr2=0x7f76bc1bcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 shutdown_connections 2024-04-16T07:50:14.258 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:14.257+0000 7f76c14ad640 1 -- 172.21.15.130:0/734402159 wait complete. 2024-04-16T07:50:14.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2024-04-16T07:50:14.268 DEBUG:teuthology.parallel:result is None 2024-04-16T07:50:15.211 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-16T07:50:15.253 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-16T07:50:15.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.325+0000 7f4d4be6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:15.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.331+0000 7f4d4be6d640 1 Processor -- start 2024-04-16T07:50:15.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.331+0000 7f4d4be6d640 1 -- start start 2024-04-16T07:50:15.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.332+0000 7f4d4be6d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f4d4409af10 0x7f4d440a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.332+0000 7f4d4be6d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d440a1c80 con 0x7f4d440a2810 2024-04-16T07:50:15.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.332+0000 7f4d4be6d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f4d440a10b0 con 0x7f4d4409af10 2024-04-16T07:50:15.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.332+0000 7f4d49c09640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f4d4409af10 0x7f4d440a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d49408640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f4d440a2810 0x7f4d44134930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42620/0 (socket says 172.21.15.130:42620) 2024-04-16T07:50:15.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d49c09640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f4d4409af10 0x7f4d440a22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58280/0 (socket says 172.21.15.130:58280) 2024-04-16T07:50:15.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d49408640 1 -- 172.21.15.130:0/2228334333 learned_addr learned my addr 172.21.15.130:0/2228334333 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d49c09640 1 -- 172.21.15.130:0/2228334333 >> v1:172.21.15.130:6789/0 conn(0x7f4d440a2810 legacy=0x7f4d44134930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d49c09640 1 -- 172.21.15.130:0/2228334333 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d440a1970 con 0x7f4d4409af10 2024-04-16T07:50:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d48c07640 1 -- 172.21.15.130:0/2228334333 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1163779142 0 0) 0x7f4d440a1c80 con 0x7f4d440a2810 2024-04-16T07:50:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d49c09640 1 --2- 172.21.15.130:0/2228334333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d440a22d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbf9de4bb334932 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d48c07640 1 -- 172.21.15.130:0/2228334333 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d40009d90 con 0x7f4d4409af10 2024-04-16T07:50:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d48c07640 1 -- 172.21.15.130:0/2228334333 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d4000e230 con 0x7f4d4409af10 2024-04-16T07:50:15.334 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.333+0000 7f4d48c07640 1 -- 172.21.15.130:0/2228334333 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d4000e4d0 con 0x7f4d4409af10 2024-04-16T07:50:15.335 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.334+0000 7f4d4be6d640 1 -- 172.21.15.130:0/2228334333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 msgr2=0x7f4d440a22d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.335 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.334+0000 7f4d4be6d640 1 --2- 172.21.15.130:0/2228334333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d440a22d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.334+0000 7f4d4be6d640 1 -- 172.21.15.130:0/2228334333 shutdown_connections 2024-04-16T07:50:15.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.334+0000 7f4d4be6d640 1 --2- 172.21.15.130:0/2228334333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d440a22d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.334+0000 7f4d4be6d640 1 -- 172.21.15.130:0/2228334333 >> 172.21.15.130:0/2228334333 conn(0x7f4d440a3c90 msgr2=0x7f4d440a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:15.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.334+0000 7f4d4be6d640 1 -- 172.21.15.130:0/2228334333 shutdown_connections 2024-04-16T07:50:15.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.335+0000 7f4d4be6d640 1 -- 172.21.15.130:0/2228334333 wait complete. 2024-04-16T07:50:15.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.335+0000 7f4d4be6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:15.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.336+0000 7f4d4be6d640 1 Processor -- start 2024-04-16T07:50:15.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.336+0000 7f4d4be6d640 1 -- start start 2024-04-16T07:50:15.338 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.338+0000 7f4d4be6d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d4411dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.338 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.338+0000 7f4d4be6d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4411f050 con 0x7f4d4409af10 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.338+0000 7f4d49c09640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d4411dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.338+0000 7f4d49c09640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d4411dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58282/0 (socket says 172.21.15.130:58282) 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.338+0000 7f4d49c09640 1 -- 172.21.15.130:0/667093538 learned_addr learned my addr 172.21.15.130:0/667093538 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.338+0000 7f4d49c09640 1 -- 172.21.15.130:0/667093538 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d44121a70 con 0x7f4d4409af10 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.338+0000 7f4d49c09640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d4411dfb0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.339+0000 7f4d3a7fc640 1 -- 172.21.15.130:0/667093538 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d40013610 con 0x7f4d4409af10 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.339+0000 7f4d3a7fc640 1 -- 172.21.15.130:0/667093538 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d40013770 con 0x7f4d4409af10 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.339+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d44121ca0 con 0x7f4d4409af10 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.339+0000 7f4d3a7fc640 1 -- 172.21.15.130:0/667093538 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d40013a10 con 0x7f4d4409af10 2024-04-16T07:50:15.339 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.339+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d44120c30 con 0x7f4d4409af10 2024-04-16T07:50:15.340 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.340+0000 7f4d3a7fc640 1 -- 172.21.15.130:0/667093538 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f4d40013bc0 con 0x7f4d4409af10 2024-04-16T07:50:15.340 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.340+0000 7f4d3a7fc640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f4d28038980 0x7f4d2803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.340 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.340+0000 7f4d3a7fc640 1 -- 172.21.15.130:0/667093538 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f4d40018040 con 0x7f4d4409af10 2024-04-16T07:50:15.340 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.340+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d441208d0 con 0x7f4d4409af10 2024-04-16T07:50:15.341 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.340+0000 7f4d49408640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f4d28038980 0x7f4d2803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.341 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.341+0000 7f4d49408640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f4d28038980 0x7f4d2803ae90 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.345 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.346+0000 7f4d3a7fc640 1 -- 172.21.15.130:0/667093538 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f4d40011380 con 0x7f4d4409af10 2024-04-16T07:50:15.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.367+0000 7f81ace17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:15.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.373+0000 7f81ace17640 1 Processor -- start 2024-04-16T07:50:15.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.373+0000 7f81ace17640 1 -- start start 2024-04-16T07:50:15.374 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f81ace17640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f81a8137ca0 0x7f81a813a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.374 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f81ace17640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a80a1c80 con 0x7f81a809af10 2024-04-16T07:50:15.374 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f81ace17640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f81a80a10b0 con 0x7f81a8137ca0 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f819ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f81a8137ca0 0x7f81a813a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f81a659c640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f81a809af10 0x7f81a80a22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42626/0 (socket says 172.21.15.130:42626) 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f81a659c640 1 -- 172.21.15.130:0/3734831369 learned_addr learned my addr 172.21.15.130:0/3734831369 (peer_addr_for_me v1:172.21.15.130:0/0) 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f819ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f81a8137ca0 0x7f81a813a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58298/0 (socket says 172.21.15.130:58298) 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f819ffff640 1 -- 172.21.15.130:0/3734831369 >> v1:172.21.15.130:6789/0 conn(0x7f81a809af10 legacy=0x7f81a80a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f819ffff640 1 -- 172.21.15.130:0/3734831369 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a80a1970 con 0x7f81a8137ca0 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.374+0000 7f81a5d9b640 1 -- 172.21.15.130:0/3734831369 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577973049 0 0) 0x7f81a80a1c80 con 0x7f81a809af10 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.375+0000 7f819ffff640 1 --2- 172.21.15.130:0/3734831369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a8137ca0 0x7f81a813a0c0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28622fe5eae5ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.375+0000 7f81a5d9b640 1 -- 172.21.15.130:0/3734831369 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8190009d90 con 0x7f81a8137ca0 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.375+0000 7f81a5d9b640 1 -- 172.21.15.130:0/3734831369 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f819000e230 con 0x7f81a8137ca0 2024-04-16T07:50:15.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.375+0000 7f81a5d9b640 1 -- 172.21.15.130:0/3734831369 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f819000e4d0 con 0x7f81a8137ca0 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.375+0000 7f81ace17640 1 -- 172.21.15.130:0/3734831369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a8137ca0 msgr2=0x7f81a813a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.375+0000 7f81ace17640 1 --2- 172.21.15.130:0/3734831369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a8137ca0 0x7f81a813a0c0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.376+0000 7f81ace17640 1 -- 172.21.15.130:0/3734831369 shutdown_connections 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.376+0000 7f81ace17640 1 --2- 172.21.15.130:0/3734831369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a8137ca0 0x7f81a813a0c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.376+0000 7f81ace17640 1 -- 172.21.15.130:0/3734831369 >> 172.21.15.130:0/3734831369 conn(0x7f81a80a3c90 msgr2=0x7f81a80a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.376+0000 7f81ace17640 1 -- 172.21.15.130:0/3734831369 shutdown_connections 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.376+0000 7f81ace17640 1 -- 172.21.15.130:0/3734831369 wait complete. 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.376+0000 7f81ace17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:15.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.377+0000 7f81ace17640 1 Processor -- start 2024-04-16T07:50:15.378 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.377+0000 7f81ace17640 1 -- start start 2024-04-16T07:50:15.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.378+0000 7f81ace17640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a809af10 0x7f81a811f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.378+0000 7f81ace17640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81209f0 con 0x7f81a809af10 2024-04-16T07:50:15.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.378+0000 7f81a659c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a809af10 0x7f81a811f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.378+0000 7f81a659c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a809af10 0x7f81a811f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58304/0 (socket says 172.21.15.130:58304) 2024-04-16T07:50:15.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.378+0000 7f81a659c640 1 -- 172.21.15.130:0/3924906215 learned_addr learned my addr 172.21.15.130:0/3924906215 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:15.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.379+0000 7f81a659c640 1 -- 172.21.15.130:0/3924906215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a811ee60 con 0x7f81a809af10 2024-04-16T07:50:15.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.379+0000 7f81a659c640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a809af10 0x7f81a811f960 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.379+0000 7f819effd640 1 -- 172.21.15.130:0/3924906215 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8198004b80 con 0x7f81a809af10 2024-04-16T07:50:15.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.379+0000 7f819effd640 1 -- 172.21.15.130:0/3924906215 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8198004ce0 con 0x7f81a809af10 2024-04-16T07:50:15.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.379+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a8122e30 con 0x7f81a809af10 2024-04-16T07:50:15.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.379+0000 7f819effd640 1 -- 172.21.15.130:0/3924906215 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8198007120 con 0x7f81a809af10 2024-04-16T07:50:15.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.379+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a811f090 con 0x7f81a809af10 2024-04-16T07:50:15.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.380+0000 7f819effd640 1 -- 172.21.15.130:0/3924906215 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f81980072d0 con 0x7f81a809af10 2024-04-16T07:50:15.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.380+0000 7f819effd640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f8188038980 0x7f818803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.380+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a8063930 con 0x7f81a809af10 2024-04-16T07:50:15.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.380+0000 7f819effd640 1 -- 172.21.15.130:0/3924906215 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f8198048310 con 0x7f81a809af10 2024-04-16T07:50:15.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.381+0000 7f819ffff640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f8188038980 0x7f818803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.381+0000 7f819ffff640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f8188038980 0x7f818803ae90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.386+0000 7f819effd640 1 -- 172.21.15.130:0/3924906215 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f819800f230 con 0x7f81a809af10 2024-04-16T07:50:15.532 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.532+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4d44063930 con 0x7f4d4409af10 2024-04-16T07:50:15.532 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.533+0000 7f4d3a7fc640 1 -- 172.21.15.130:0/667093538 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f4d40001d40 con 0x7f4d4409af10 2024-04-16T07:50:15.532 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:15.535 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f4d28038980 msgr2=0x7f4d2803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.535 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f4d28038980 0x7f4d2803ae90 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.535 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 msgr2=0x7f4d4411dfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.535 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d4411dfb0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.535 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 shutdown_connections 2024-04-16T07:50:15.535 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f4d28038980 0x7f4d2803ae90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.535 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 --2- 172.21.15.130:0/667093538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d4409af10 0x7f4d4411dfb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.536 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 >> 172.21.15.130:0/667093538 conn(0x7f4d440a3c90 msgr2=0x7f4d441bc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:15.536 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 shutdown_connections 2024-04-16T07:50:15.536 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.535+0000 7f4d4be6d640 1 -- 172.21.15.130:0/667093538 wait complete. 2024-04-16T07:50:15.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2024-04-16T07:50:15.546 DEBUG:teuthology.parallel:result is None 2024-04-16T07:50:15.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.570+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f81a80008d0 con 0x7f81a809af10 2024-04-16T07:50:15.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.570+0000 7f819effd640 1 -- 172.21.15.130:0/3924906215 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f8198040e60 con 0x7f81a809af10 2024-04-16T07:50:15.570 INFO:teuthology.orchestra.run.smithi130.stdout:38654705670 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f8188038980 msgr2=0x7f818803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f8188038980 0x7f818803ae90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a809af10 msgr2=0x7f81a811f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a809af10 0x7f81a811f960 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 shutdown_connections 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f8188038980 0x7f818803ae90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 --2- 172.21.15.130:0/3924906215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81a809af10 0x7f81a811f960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 >> 172.21.15.130:0/3924906215 conn(0x7f81a80a3c90 msgr2=0x7f81a81347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 shutdown_connections 2024-04-16T07:50:15.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.572+0000 7f81ace17640 1 -- 172.21.15.130:0/3924906215 wait complete. 2024-04-16T07:50:15.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2024-04-16T07:50:15.583 DEBUG:teuthology.parallel:result is None 2024-04-16T07:50:15.583 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-16T07:50:15.584 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T07:50:15.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.732+0000 7f0e864ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:15.737 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.737+0000 7f0e864ed640 1 Processor -- start 2024-04-16T07:50:15.737 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.738+0000 7f0e864ed640 1 -- start start 2024-04-16T07:50:15.740 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e864ed640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f0e80134a70 0x7f0e80136e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.740 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e864ed640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e8009cd60 con 0x7f0e80131c50 2024-04-16T07:50:15.740 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e864ed640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f0e80131800 con 0x7f0e80134a70 2024-04-16T07:50:15.740 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e77fff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f0e80134a70 0x7f0e80136e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.740 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e77fff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f0e80134a70 0x7f0e80136e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58310/0 (socket says 172.21.15.130:58310) 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e7ffff640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f0e80131c50 0x7f0e80132060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42634/0 (socket says 172.21.15.130:42634) 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e77fff640 1 -- 172.21.15.130:0/1381973014 learned_addr learned my addr 172.21.15.130:0/1381973014 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e77fff640 1 -- 172.21.15.130:0/1381973014 >> v1:172.21.15.130:6789/0 conn(0x7f0e80131c50 legacy=0x7f0e80132060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e77fff640 1 -- 172.21.15.130:0/1381973014 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8009b070 con 0x7f0e80134a70 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e7f7fe640 1 -- 172.21.15.130:0/1381973014 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1087875374 0 0) 0x7f0e8009cd60 con 0x7f0e80131c50 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e77fff640 1 --2- 172.21.15.130:0/1381973014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80134a70 0x7f0e80136e90 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3826f7faac29dc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e7f7fe640 1 -- 172.21.15.130:0/1381973014 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0e68010570 con 0x7f0e80134a70 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e7f7fe640 1 -- 172.21.15.130:0/1381973014 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0e680106d0 con 0x7f0e80134a70 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.739+0000 7f0e7f7fe640 1 -- 172.21.15.130:0/1381973014 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0e68010970 con 0x7f0e80134a70 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.740+0000 7f0e864ed640 1 -- 172.21.15.130:0/1381973014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80134a70 msgr2=0x7f0e80136e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.740+0000 7f0e864ed640 1 --2- 172.21.15.130:0/1381973014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80134a70 0x7f0e80136e90 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.741 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.740+0000 7f0e864ed640 1 -- 172.21.15.130:0/1381973014 shutdown_connections 2024-04-16T07:50:15.742 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.740+0000 7f0e864ed640 1 --2- 172.21.15.130:0/1381973014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80134a70 0x7f0e80136e90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.742 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.740+0000 7f0e864ed640 1 -- 172.21.15.130:0/1381973014 >> 172.21.15.130:0/1381973014 conn(0x7f0e800a0a30 msgr2=0x7f0e800a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:15.742 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.740+0000 7f0e864ed640 1 -- 172.21.15.130:0/1381973014 shutdown_connections 2024-04-16T07:50:15.742 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.740+0000 7f0e864ed640 1 -- 172.21.15.130:0/1381973014 wait complete. 2024-04-16T07:50:15.742 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.741+0000 7f0e864ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.742+0000 7f0e864ed640 1 Processor -- start 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.742+0000 7f0e864ed640 1 -- start start 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e864ed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80131c50 0x7f0e801c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e864ed640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8011ab50 con 0x7f0e80131c50 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80131c50 0x7f0e801c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80131c50 0x7f0e801c5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58326/0 (socket says 172.21.15.130:58326) 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e7ffff640 1 -- 172.21.15.130:0/2521642587 learned_addr learned my addr 172.21.15.130:0/2521642587 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e7ffff640 1 -- 172.21.15.130:0/2521642587 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8011c790 con 0x7f0e80131c50 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e7ffff640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80131c50 0x7f0e801c5110 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e7d7fa640 1 -- 172.21.15.130:0/2521642587 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0e70004b80 con 0x7f0e80131c50 2024-04-16T07:50:15.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e7d7fa640 1 -- 172.21.15.130:0/2521642587 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0e70004ce0 con 0x7f0e80131c50 2024-04-16T07:50:15.745 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.743+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8011ad80 con 0x7f0e80131c50 2024-04-16T07:50:15.745 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.744+0000 7f0e7d7fa640 1 -- 172.21.15.130:0/2521642587 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0e70007120 con 0x7f0e80131c50 2024-04-16T07:50:15.745 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.744+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8011e020 con 0x7f0e80131c50 2024-04-16T07:50:15.746 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.744+0000 7f0e7d7fa640 1 -- 172.21.15.130:0/2521642587 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f0e700072d0 con 0x7f0e80131c50 2024-04-16T07:50:15.746 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.744+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8011d0d0 con 0x7f0e80131c50 2024-04-16T07:50:15.746 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.745+0000 7f0e7d7fa640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0e600389b0 0x7f0e6003aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:15.746 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.745+0000 7f0e7d7fa640 1 -- 172.21.15.130:0/2521642587 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f0e70048310 con 0x7f0e80131c50 2024-04-16T07:50:15.746 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.745+0000 7f0e77fff640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0e600389b0 0x7f0e6003aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:15.746 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.746+0000 7f0e77fff640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0e600389b0 0x7f0e6003aec0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:15.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.750+0000 7f0e7d7fa640 1 -- 172.21.15.130:0/2521642587 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f0e7000a9e0 con 0x7f0e80131c50 2024-04-16T07:50:15.923 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.923+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 --> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e8011ed40 con 0x7f0e600389b0 2024-04-16T07:50:15.925 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.925+0000 7f0e7d7fa640 1 -- 172.21.15.130:0/2521642587 <== mgr.4100 v2:172.21.15.164:6800/4054838717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+29954 (crc 0 0 0) 0x7f0e8011ed40 con 0x7f0e600389b0 2024-04-16T07:50:15.925 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:50:15.926 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-04-16T07:50:15.927 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0e600389b0 msgr2=0x7f0e6003aec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.927 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0e600389b0 0x7f0e6003aec0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.927 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80131c50 msgr2=0x7f0e801c5110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:15.927 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80131c50 0x7f0e801c5110 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.927 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 shutdown_connections 2024-04-16T07:50:15.928 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f0e600389b0 0x7f0e6003aec0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.928 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 --2- 172.21.15.130:0/2521642587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0e80131c50 0x7f0e801c5110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:15.928 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 >> 172.21.15.130:0/2521642587 conn(0x7f0e800a0a30 msgr2=0x7f0e8009d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:15.928 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 shutdown_connections 2024-04-16T07:50:15.928 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:15.927+0000 7f0e864ed640 1 -- 172.21.15.130:0/2521642587 wait complete. 2024-04-16T07:50:15.938 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-16T07:50:14.165781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.744142"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.288971+0000","last_change":"2024-04-16T07:50:11.290117+0000","last_active":"2024-04-16T07:50:11.288971+0000","last_peered":"2024-04-16T07:50:11.288971+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.284536+0000","last_became_peered":"2024-04-16T07:50:11.284536+0000","last_unstale":"2024-04-16T07:50:11.288971+0000","last_undegraded":"2024-04-16T07:50:11.288971+0000","last_fullsized":"2024-04-16T07:50:11.288971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.302602+0000","last_change":"2024-04-16T07:50:11.303365+0000","last_active":"2024-04-16T07:50:11.302602+0000","last_peered":"2024-04-16T07:50:11.302602+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.299161+0000","last_became_peered":"2024-04-16T07:50:11.299161+0000","last_unstale":"2024-04-16T07:50:11.302602+0000","last_undegraded":"2024-04-16T07:50:11.302602+0000","last_fullsized":"2024-04-16T07:50:11.302602+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T07:50:12.245550+0000","last_change":"2024-04-16T07:50:11.275896+0000","last_active":"2024-04-16T07:50:12.245550+0000","last_peered":"2024-04-16T07:50:12.245550+0000","last_clean":"2024-04-16T07:50:12.245550+0000","last_became_active":"2024-04-16T07:50:11.270629+0000","last_became_peered":"2024-04-16T07:50:11.270629+0000","last_unstale":"2024-04-16T07:50:12.245550+0000","last_undegraded":"2024-04-16T07:50:12.245550+0000","last_fullsized":"2024-04-16T07:50:12.245550+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.283604+0000","last_change":"2024-04-16T07:50:11.284996+0000","last_active":"2024-04-16T07:50:11.283604+0000","last_peered":"2024-04-16T07:50:11.283604+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.277953+0000","last_became_peered":"2024-04-16T07:50:11.277953+0000","last_unstale":"2024-04-16T07:50:11.283604+0000","last_undegraded":"2024-04-16T07:50:11.283604+0000","last_fullsized":"2024-04-16T07:50:11.283604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.279631+0000","last_change":"2024-04-16T07:50:11.280600+0000","last_active":"2024-04-16T07:50:11.279631+0000","last_peered":"2024-04-16T07:50:11.279631+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.275870+0000","last_became_peered":"2024-04-16T07:50:11.275870+0000","last_unstale":"2024-04-16T07:50:11.279631+0000","last_undegraded":"2024-04-16T07:50:11.279631+0000","last_fullsized":"2024-04-16T07:50:11.279631+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.279189+0000","last_change":"2024-04-16T07:50:11.280370+0000","last_active":"2024-04-16T07:50:11.279189+0000","last_peered":"2024-04-16T07:50:11.279189+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.274490+0000","last_became_peered":"2024-04-16T07:50:11.274490+0000","last_unstale":"2024-04-16T07:50:11.279189+0000","last_undegraded":"2024-04-16T07:50:11.279189+0000","last_fullsized":"2024-04-16T07:50:11.279189+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.285360+0000","last_change":"2024-04-16T07:50:11.286401+0000","last_active":"2024-04-16T07:50:11.285360+0000","last_peered":"2024-04-16T07:50:11.285360+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.282031+0000","last_became_peered":"2024-04-16T07:50:11.282031+0000","last_unstale":"2024-04-16T07:50:11.285360+0000","last_undegraded":"2024-04-16T07:50:11.285360+0000","last_fullsized":"2024-04-16T07:50:11.285360+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.299228+0000","last_change":"2024-04-16T07:50:11.300870+0000","last_active":"2024-04-16T07:50:11.299228+0000","last_peered":"2024-04-16T07:50:11.299228+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.294800+0000","last_became_peered":"2024-04-16T07:50:11.294800+0000","last_unstale":"2024-04-16T07:50:11.299228+0000","last_undegraded":"2024-04-16T07:50:11.299228+0000","last_fullsized":"2024-04-16T07:50:11.299228+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T07:50:12.242058+0000","last_change":"2024-04-16T07:50:09.262214+0000","last_active":"2024-04-16T07:50:12.242058+0000","last_peered":"2024-04-16T07:50:12.242058+0000","last_clean":"2024-04-16T07:50:12.242058+0000","last_became_active":"2024-04-16T07:50:09.256726+0000","last_became_peered":"2024-04-16T07:50:09.256726+0000","last_unstale":"2024-04-16T07:50:12.242058+0000","last_undegraded":"2024-04-16T07:50:12.242058+0000","last_fullsized":"2024-04-16T07:50:12.242058+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:08.208873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:08.208873+0000","last_clean_scrub_stamp":"2024-04-16T07:50:08.208873+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-16T07:50:15.939 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T07:50:16.051 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.048+0000 7f7d62e06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:16.052 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.053+0000 7f7d62e06640 1 Processor -- start 2024-04-16T07:50:16.052 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.053+0000 7f7d62e06640 1 -- start start 2024-04-16T07:50:16.054 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d62e06640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f7d5c131c20 0x7f7d5c132030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d62e06640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d5c1319a0 con 0x7f7d5c1349e0 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d62e06640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f7d5c0a2b70 con 0x7f7d5c131c20 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d60ba2640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f7d5c131c20 0x7f7d5c132030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d60ba2640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f7d5c131c20 0x7f7d5c132030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58338/0 (socket says 172.21.15.130:58338) 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d5bfff640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f7d5c1349e0 0x7f7d5c134df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42638/0 (socket says 172.21.15.130:42638) 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d60ba2640 1 -- 172.21.15.130:0/1323941404 learned_addr learned my addr 172.21.15.130:0/1323941404 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d60ba2640 1 -- 172.21.15.130:0/1323941404 >> v1:172.21.15.130:6789/0 conn(0x7f7d5c1349e0 legacy=0x7f7d5c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d5b7fe640 1 -- 172.21.15.130:0/1323941404 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662887969 0 0) 0x7f7d5c1319a0 con 0x7f7d5c1349e0 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.054+0000 7f7d60ba2640 1 -- 172.21.15.130:0/1323941404 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c09e570 con 0x7f7d5c131c20 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d60ba2640 1 --2- 172.21.15.130:0/1323941404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c132030 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50cc6ce3f709ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d5b7fe640 1 -- 172.21.15.130:0/1323941404 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d54009e50 con 0x7f7d5c131c20 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d5b7fe640 1 -- 172.21.15.130:0/1323941404 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7d5400e230 con 0x7f7d5c131c20 2024-04-16T07:50:16.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d5b7fe640 1 -- 172.21.15.130:0/1323941404 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d5400e4d0 con 0x7f7d5c131c20 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d62e06640 1 -- 172.21.15.130:0/1323941404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 msgr2=0x7f7d5c132030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d62e06640 1 --2- 172.21.15.130:0/1323941404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c132030 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d62e06640 1 -- 172.21.15.130:0/1323941404 shutdown_connections 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d62e06640 1 --2- 172.21.15.130:0/1323941404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c132030 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.055+0000 7f7d62e06640 1 -- 172.21.15.130:0/1323941404 >> 172.21.15.130:0/1323941404 conn(0x7f7d5c0a3d50 msgr2=0x7f7d5c0a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.056+0000 7f7d62e06640 1 -- 172.21.15.130:0/1323941404 shutdown_connections 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.056+0000 7f7d62e06640 1 -- 172.21.15.130:0/1323941404 wait complete. 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.056+0000 7f7d62e06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.057+0000 7f7d62e06640 1 Processor -- start 2024-04-16T07:50:16.057 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.057+0000 7f7d62e06640 1 -- start start 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.058+0000 7f7d62e06640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c11d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.058+0000 7f7d62e06640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c121080 con 0x7f7d5c131c20 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.058+0000 7f7d60ba2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c11d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.058+0000 7f7d60ba2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c11d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58342/0 (socket says 172.21.15.130:58342) 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.058+0000 7f7d60ba2640 1 -- 172.21.15.130:0/722234737 learned_addr learned my addr 172.21.15.130:0/722234737 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.059+0000 7f7d60ba2640 1 -- 172.21.15.130:0/722234737 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c11e450 con 0x7f7d5c131c20 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.059+0000 7f7d60ba2640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c11d330 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.059+0000 7f7d597fa640 1 -- 172.21.15.130:0/722234737 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d5400edc0 con 0x7f7d5c131c20 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.059+0000 7f7d597fa640 1 -- 172.21.15.130:0/722234737 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7d54015420 con 0x7f7d5c131c20 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.059+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c11f450 con 0x7f7d5c131c20 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.059+0000 7f7d597fa640 1 -- 172.21.15.130:0/722234737 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7d540156c0 con 0x7f7d5c131c20 2024-04-16T07:50:16.059 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.059+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c120580 con 0x7f7d5c131c20 2024-04-16T07:50:16.060 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.060+0000 7f7d597fa640 1 -- 172.21.15.130:0/722234737 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f7d540158c0 con 0x7f7d5c131c20 2024-04-16T07:50:16.060 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.060+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c11e640 con 0x7f7d5c131c20 2024-04-16T07:50:16.060 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.060+0000 7f7d597fa640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d3c038960 0x7f7d3c03ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:16.061 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.060+0000 7f7d597fa640 1 -- 172.21.15.130:0/722234737 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f7d54049810 con 0x7f7d5c131c20 2024-04-16T07:50:16.061 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.060+0000 7f7d5bfff640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d3c038960 0x7f7d3c03ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:16.061 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.061+0000 7f7d5bfff640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d3c038960 0x7f7d3c03ae70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:16.065 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.065+0000 7f7d597fa640 1 -- 172.21.15.130:0/722234737 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f7d54010a00 con 0x7f7d5c131c20 2024-04-16T07:50:16.240 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.240+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 --> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d5c09e250 con 0x7f7d3c038960 2024-04-16T07:50:16.241 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.241+0000 7f7d597fa640 1 -- 172.21.15.130:0/722234737 <== mgr.4100 v2:172.21.15.164:6800/4054838717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+29954 (crc 0 0 0) 0x7f7d4c001d70 con 0x7f7d3c038960 2024-04-16T07:50:16.242 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:50:16.242 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-04-16T07:50:16.243 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d3c038960 msgr2=0x7f7d3c03ae70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d3c038960 0x7f7d3c03ae70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 msgr2=0x7f7d5c11d330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c11d330 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 shutdown_connections 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f7d3c038960 0x7f7d3c03ae70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 --2- 172.21.15.130:0/722234737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d5c131c20 0x7f7d5c11d330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 >> 172.21.15.130:0/722234737 conn(0x7f7d5c0a3d50 msgr2=0x7f7d5c0a15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 shutdown_connections 2024-04-16T07:50:16.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.244+0000 7f7d62e06640 1 -- 172.21.15.130:0/722234737 wait complete. 2024-04-16T07:50:16.254 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-04-16T07:50:16.166279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.744640"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.288971+0000","last_change":"2024-04-16T07:50:11.290117+0000","last_active":"2024-04-16T07:50:11.288971+0000","last_peered":"2024-04-16T07:50:11.288971+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.284536+0000","last_became_peered":"2024-04-16T07:50:11.284536+0000","last_unstale":"2024-04-16T07:50:11.288971+0000","last_undegraded":"2024-04-16T07:50:11.288971+0000","last_fullsized":"2024-04-16T07:50:11.288971+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.302602+0000","last_change":"2024-04-16T07:50:11.303365+0000","last_active":"2024-04-16T07:50:11.302602+0000","last_peered":"2024-04-16T07:50:11.302602+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.299161+0000","last_became_peered":"2024-04-16T07:50:11.299161+0000","last_unstale":"2024-04-16T07:50:11.302602+0000","last_undegraded":"2024-04-16T07:50:11.302602+0000","last_fullsized":"2024-04-16T07:50:11.302602+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T07:50:12.245550+0000","last_change":"2024-04-16T07:50:11.275896+0000","last_active":"2024-04-16T07:50:12.245550+0000","last_peered":"2024-04-16T07:50:12.245550+0000","last_clean":"2024-04-16T07:50:12.245550+0000","last_became_active":"2024-04-16T07:50:11.270629+0000","last_became_peered":"2024-04-16T07:50:11.270629+0000","last_unstale":"2024-04-16T07:50:12.245550+0000","last_undegraded":"2024-04-16T07:50:12.245550+0000","last_fullsized":"2024-04-16T07:50:12.245550+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.283604+0000","last_change":"2024-04-16T07:50:11.284996+0000","last_active":"2024-04-16T07:50:11.283604+0000","last_peered":"2024-04-16T07:50:11.283604+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.277953+0000","last_became_peered":"2024-04-16T07:50:11.277953+0000","last_unstale":"2024-04-16T07:50:11.283604+0000","last_undegraded":"2024-04-16T07:50:11.283604+0000","last_fullsized":"2024-04-16T07:50:11.283604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.279631+0000","last_change":"2024-04-16T07:50:11.280600+0000","last_active":"2024-04-16T07:50:11.279631+0000","last_peered":"2024-04-16T07:50:11.279631+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.275870+0000","last_became_peered":"2024-04-16T07:50:11.275870+0000","last_unstale":"2024-04-16T07:50:11.279631+0000","last_undegraded":"2024-04-16T07:50:11.279631+0000","last_fullsized":"2024-04-16T07:50:11.279631+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.279189+0000","last_change":"2024-04-16T07:50:11.280370+0000","last_active":"2024-04-16T07:50:11.279189+0000","last_peered":"2024-04-16T07:50:11.279189+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.274490+0000","last_became_peered":"2024-04-16T07:50:11.274490+0000","last_unstale":"2024-04-16T07:50:11.279189+0000","last_undegraded":"2024-04-16T07:50:11.279189+0000","last_fullsized":"2024-04-16T07:50:11.279189+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.285360+0000","last_change":"2024-04-16T07:50:11.286401+0000","last_active":"2024-04-16T07:50:11.285360+0000","last_peered":"2024-04-16T07:50:11.285360+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.282031+0000","last_became_peered":"2024-04-16T07:50:11.282031+0000","last_unstale":"2024-04-16T07:50:11.285360+0000","last_undegraded":"2024-04-16T07:50:11.285360+0000","last_fullsized":"2024-04-16T07:50:11.285360+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T07:50:11.299228+0000","last_change":"2024-04-16T07:50:11.300870+0000","last_active":"2024-04-16T07:50:11.299228+0000","last_peered":"2024-04-16T07:50:11.299228+0000","last_clean":"2024-04-16T07:50:10.217408+0000","last_became_active":"2024-04-16T07:50:11.294800+0000","last_became_peered":"2024-04-16T07:50:11.294800+0000","last_unstale":"2024-04-16T07:50:11.299228+0000","last_undegraded":"2024-04-16T07:50:11.299228+0000","last_fullsized":"2024-04-16T07:50:11.299228+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:10.217408+0000","last_clean_scrub_stamp":"2024-04-16T07:50:10.217408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T07:50:12.242058+0000","last_change":"2024-04-16T07:50:09.262214+0000","last_active":"2024-04-16T07:50:12.242058+0000","last_peered":"2024-04-16T07:50:12.242058+0000","last_clean":"2024-04-16T07:50:12.242058+0000","last_became_active":"2024-04-16T07:50:09.256726+0000","last_became_peered":"2024-04-16T07:50:09.256726+0000","last_unstale":"2024-04-16T07:50:12.242058+0000","last_undegraded":"2024-04-16T07:50:12.242058+0000","last_fullsized":"2024-04-16T07:50:12.242058+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T07:50:08.208873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T07:50:08.208873+0000","last_clean_scrub_stamp":"2024-04-16T07:50:08.208873+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-16T07:50:16.255 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-16T07:50:16.255 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-16T07:50:16.255 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-16T07:50:16.255 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-16T07:50:16.366 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.364+0000 7f5ffab9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:16.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.368+0000 7f5ffab9d640 1 Processor -- start 2024-04-16T07:50:16.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.368+0000 7f5ffab9d640 1 -- start start 2024-04-16T07:50:16.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ffab9d640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f5ff4132030 0x7f5ff41350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:16.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ffab9d640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff4132440 con 0x7f5ff4131c20 2024-04-16T07:50:16.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ffab9d640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7f5ff40a2b70 con 0x7f5ff4132030 2024-04-16T07:50:16.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ff3fff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f5ff4132030 0x7f5ff41350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:16.371 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ff3fff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7f5ff4132030 0x7f5ff41350f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58344/0 (socket says 172.21.15.130:58344) 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ff3fff640 1 -- 172.21.15.130:0/609332237 learned_addr learned my addr 172.21.15.130:0/609332237 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ff8939640 1 --1- >> v1:172.21.15.130:6789/0 conn(0x7f5ff4131c20 0x7f5ff41349e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.130:6789/0 says I am v1:172.21.15.130:42648/0 (socket says 172.21.15.130:42648) 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ff3fff640 1 -- 172.21.15.130:0/609332237 >> v1:172.21.15.130:6789/0 conn(0x7f5ff4131c20 legacy=0x7f5ff41349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ff3fff640 1 -- 172.21.15.130:0/609332237 --> v2:172.21.15.130:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff41319a0 con 0x7f5ff4132030 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.369+0000 7f5ff37fe640 1 -- 172.21.15.130:0/609332237 <== mon.0 v1:172.21.15.130:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415257244 0 0) 0x7f5ff4132440 con 0x7f5ff4131c20 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.370+0000 7f5ff3fff640 1 --2- 172.21.15.130:0/609332237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4132030 0x7f5ff41350f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb60e261fe0ec7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.370+0000 7f5ff37fe640 1 -- 172.21.15.130:0/609332237 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5fe0009e50 con 0x7f5ff4132030 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.370+0000 7f5ff37fe640 1 -- 172.21.15.130:0/609332237 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5fe000e230 con 0x7f5ff4132030 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.370+0000 7f5ff37fe640 1 -- 172.21.15.130:0/609332237 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5fe000e4d0 con 0x7f5ff4132030 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 1 -- 172.21.15.130:0/609332237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4132030 msgr2=0x7f5ff41350f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 1 --2- 172.21.15.130:0/609332237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4132030 0x7f5ff41350f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 1 -- 172.21.15.130:0/609332237 shutdown_connections 2024-04-16T07:50:16.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 1 --2- 172.21.15.130:0/609332237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4132030 0x7f5ff41350f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 1 -- 172.21.15.130:0/609332237 >> 172.21.15.130:0/609332237 conn(0x7f5ff40a3d50 msgr2=0x7f5ff40a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:16.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 1 -- 172.21.15.130:0/609332237 shutdown_connections 2024-04-16T07:50:16.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 1 -- 172.21.15.130:0/609332237 wait complete. 2024-04-16T07:50:16.373 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.371+0000 7f5ffab9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:50:16.375 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.372+0000 7f5ffab9d640 1 Processor -- start 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.372+0000 7f5ffab9d640 1 -- start start 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.373+0000 7f5ffab9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4131c20 0x7f5ff411d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.373+0000 7f5ffab9d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4074300 con 0x7f5ff4131c20 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff8939640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4131c20 0x7f5ff411d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff8939640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4131c20 0x7f5ff411d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58350/0 (socket says 172.21.15.130:58350) 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff8939640 1 -- 172.21.15.130:0/1516450142 learned_addr learned my addr 172.21.15.130:0/1516450142 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff8939640 1 -- 172.21.15.130:0/1516450142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff411fe40 con 0x7f5ff4131c20 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff8939640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4131c20 0x7f5ff411d330 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff17fa640 1 -- 172.21.15.130:0/1516450142 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5fe8004b80 con 0x7f5ff4131c20 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff17fa640 1 -- 172.21.15.130:0/1516450142 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5fe8004ce0 con 0x7f5ff4131c20 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff411ec40 con 0x7f5ff4131c20 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ff17fa640 1 -- 172.21.15.130:0/1516450142 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5fe8007120 con 0x7f5ff4131c20 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.374+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff411f5f0 con 0x7f5ff4131c20 2024-04-16T07:50:16.376 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.375+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff4063930 con 0x7f5ff4131c20 2024-04-16T07:50:16.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.375+0000 7f5ff17fa640 1 -- 172.21.15.130:0/1516450142 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f5fe80072d0 con 0x7f5ff4131c20 2024-04-16T07:50:16.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.375+0000 7f5ff17fa640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f5fd8038980 0x7f5fd803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:50:16.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.375+0000 7f5ff17fa640 1 -- 172.21.15.130:0/1516450142 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f5fe80482f0 con 0x7f5ff4131c20 2024-04-16T07:50:16.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.376+0000 7f5ff3fff640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f5fd8038980 0x7f5fd803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:50:16.377 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.376+0000 7f5ff3fff640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f5fd8038980 0x7f5fd803ae90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:50:16.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.381+0000 7f5ff17fa640 1 -- 172.21.15.130:0/1516450142 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (crc 0 0 0) 0x7f5fe800a9e0 con 0x7f5ff4131c20 2024-04-16T07:50:16.592 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.593+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5ff409dec0 con 0x7f5ff4131c20 2024-04-16T07:50:16.593 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.593+0000 7f5ff17fa640 1 -- 172.21.15.130:0/1516450142 <== mon.0 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (crc 0 0 0) 0x7f5fe800a5d0 con 0x7f5ff4131c20 2024-04-16T07:50:16.593 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-16T07:50:16.593 INFO:teuthology.orchestra.run.smithi130.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T07:50:16.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f5fd8038980 msgr2=0x7f5fd803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f5fd8038980 0x7f5fd803ae90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4131c20 msgr2=0x7f5ff411d330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:50:16.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4131c20 0x7f5ff411d330 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 shutdown_connections 2024-04-16T07:50:16.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7f5fd8038980 0x7f5fd803ae90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 --2- 172.21.15.130:0/1516450142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ff4131c20 0x7f5ff411d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:50:16.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 >> 172.21.15.130:0/1516450142 conn(0x7f5ff40a3d50 msgr2=0x7f5ff409ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:50:16.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 shutdown_connections 2024-04-16T07:50:16.596 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T07:50:16.595+0000 7f5ffab9d640 1 -- 172.21.15.130:0/1516450142 wait complete. 2024-04-16T07:50:16.606 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-16T07:50:16.607 INFO:teuthology.run_tasks:Running task workunit... 2024-04-16T07:50:16.619 INFO:tasks.workunit:Pulling workunits from ref a7639cb6ce633dabc4d37769d62caacb54582984 2024-04-16T07:50:16.620 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-16T07:50:16.620 DEBUG:teuthology.orchestra.run.smithi130:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-16T07:50:16.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T07:50:16.635 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-16T07:50:16.635 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-16T07:50:16.690 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-16T07:50:16.691 DEBUG:teuthology.orchestra.run.smithi130:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-16T07:50:16.745 INFO:tasks.workunit:timeout=3h 2024-04-16T07:50:16.745 INFO:tasks.workunit:cleanup=True 2024-04-16T07:50:16.746 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a7639cb6ce633dabc4d37769d62caacb54582984 2024-04-16T07:50:16.802 INFO:tasks.workunit.client.0.smithi130.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr:Note: switching to 'a7639cb6ce633dabc4d37769d62caacb54582984'. 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr: 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr:state without impacting any branches by switching back to a branch. 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr: 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr: 2024-04-16T07:51:01.944 INFO:tasks.workunit.client.0.smithi130.stderr: git switch -c 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr: 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr:Or undo this operation with: 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr: 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr: git switch - 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr: 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr: 2024-04-16T07:51:01.945 INFO:tasks.workunit.client.0.smithi130.stderr:HEAD is now at a7639cb6ce6 Merge pull request #56885 from Svelar/asan_OSDScrub 2024-04-16T07:51:01.946 DEBUG:teuthology.orchestra.run.smithi130:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-16T07:51:02.066 INFO:tasks.workunit.client.0.smithi130.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-16T07:51:02.068 INFO:tasks.workunit.client.0.smithi130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T07:51:02.068 INFO:tasks.workunit.client.0.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-16T07:51:02.806 INFO:tasks.workunit.client.0.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-16T07:51:02.868 INFO:tasks.workunit.client.0.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-16T07:51:02.910 INFO:tasks.workunit.client.0.smithi130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T07:51:02.913 INFO:tasks.workunit.client.0.smithi130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T07:51:02.913 INFO:tasks.workunit.client.0.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-16T07:51:02.951 INFO:tasks.workunit.client.0.smithi130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T07:51:02.954 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-16T07:51:02.954 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-16T07:51:02.970 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-04-16T07:51:02.971 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-04-16T07:51:02.971 DEBUG:teuthology.orchestra.run.smithi130:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7639cb6ce633dabc4d37769d62caacb54582984 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-04-16T07:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:+ parallel=1 2024-04-16T07:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' --crimson = --serial ']' 2024-04-16T07:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:+ crimson=0 2024-04-16T07:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' --crimson = --crimson ']' 2024-04-16T07:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:+ parallel=0 2024-04-16T07:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:+ crimson=1 2024-04-16T07:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:+ color= 2024-04-16T07:51:03.032 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' -t 1 ']' 2024-04-16T07:51:03.032 INFO:tasks.workunit.client.0.smithi130.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-04-16T07:51:03.032 INFO:tasks.workunit.client.0.smithi130.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-04-16T07:51:03.032 INFO:tasks.workunit.client.0.smithi130.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-04-16T07:51:03.032 INFO:tasks.workunit.client.0.smithi130.stderr:+ declare -A pids 2024-04-16T07:51:03.032 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:51:03.033 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:51:03.033 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_aio 2024-04-16T07:51:03.049 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:51:03.049 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 42 tests from 2 test suites. 2024-04-16T07:51:03.049 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:51:03.049 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 26 tests from LibRadosAio 2024-04-16T07:51:03.049 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.TooBig 2024-04-16T07:51:05.786 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.TooBig (2737 ms) 2024-04-16T07:51:05.786 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-04-16T07:51:08.800 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.SimpleWrite (3014 ms) 2024-04-16T07:51:08.800 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-04-16T07:51:11.811 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.WaitForSafe (3011 ms) 2024-04-16T07:51:11.811 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-04-16T07:51:14.828 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTrip (3018 ms) 2024-04-16T07:51:14.829 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-04-16T07:51:17.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTrip2 (3015 ms) 2024-04-16T07:51:17.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-04-16T07:51:20.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTrip3 (3019 ms) 2024-04-16T07:51:20.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-04-16T07:51:23.878 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripAppend (3016 ms) 2024-04-16T07:51:23.879 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-04-16T07:51:26.893 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RemoveTest (3014 ms) 2024-04-16T07:51:26.893 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-04-16T07:51:29.904 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3011 ms) 2024-04-16T07:51:29.905 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RmXattr 2024-04-16T07:51:32.920 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RmXattr (3015 ms) 2024-04-16T07:51:32.920 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.XattrIter 2024-04-16T07:51:35.948 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.XattrIter (3028 ms) 2024-04-16T07:51:35.948 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.IsComplete 2024-04-16T07:51:38.962 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.IsComplete (3014 ms) 2024-04-16T07:51:38.962 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.IsSafe 2024-04-16T07:51:41.975 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.IsSafe (3013 ms) 2024-04-16T07:51:41.975 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-04-16T07:51:44.986 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.ReturnValue (3011 ms) 2024-04-16T07:51:44.986 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.Flush 2024-04-16T07:51:47.997 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.Flush (3011 ms) 2024-04-16T07:51:47.998 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-04-16T07:51:51.012 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.FlushAsync (3015 ms) 2024-04-16T07:51:51.012 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-04-16T07:51:54.024 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3012 ms) 2024-04-16T07:51:54.024 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-04-16T07:51:57.038 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (3014 ms) 2024-04-16T07:51:57.038 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-04-16T07:51:59.774 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.SimpleStat (2736 ms) 2024-04-16T07:51:59.774 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-04-16T07:52:02.785 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.OperateMtime (3011 ms) 2024-04-16T07:52:02.785 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-04-16T07:52:05.798 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.Operate2Mtime (3013 ms) 2024-04-16T07:52:05.798 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-04-16T07:52:08.812 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.SimpleStatNS (3014 ms) 2024-04-16T07:52:08.812 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.StatRemove 2024-04-16T07:52:11.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.StatRemove (3011 ms) 2024-04-16T07:52:11.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-04-16T07:52:14.780 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.ExecuteClass (2958 ms) 2024-04-16T07:52:14.781 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-04-16T07:52:17.786 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.MultiWrite (3005 ms) 2024-04-16T07:52:17.786 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-04-16T07:52:20.801 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.AioUnlock (3012 ms) 2024-04-16T07:52:20.801 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 26 tests from LibRadosAio (77749 ms total) 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 16 tests from LibRadosAioEC 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1100: Skipped 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1132: Skipped 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1149: Skipped 2024-04-16T07:52:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1182: Skipped 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1215: Skipped 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1286: Skipped 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-04-16T07:52:20.803 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1326: Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1366: Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.Flush 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1385: Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1415: Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1454: Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1498: Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1532: Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1593: Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1649: Skipped 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio.cc:1681: Skipped 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (0 ms) 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 16 tests from LibRadosAioEC (0 ms total) 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 42 tests from 2 test suites ran. (77749 ms total) 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 26 tests. 2024-04-16T07:52:20.806 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 16 tests, listed below: 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-04-16T07:52:20.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-04-16T07:52:20.808 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-04-16T07:52:20.808 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:52:20.808 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:52:20.808 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_aio_pp 2024-04-16T07:52:20.817 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:52:20.817 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 57 tests from 4 test suites. 2024-04-16T07:52:20.817 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:52:20.817 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 32 tests from LibRadosAio 2024-04-16T07:52:20.818 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-04-16T07:52:23.808 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.TooBigPP (2990 ms) 2024-04-16T07:52:23.808 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-04-16T07:52:42.812 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.PoolQuotaPP (19003 ms) 2024-04-16T07:52:42.812 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-04-16T07:52:48.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.SimpleWritePP (6044 ms) 2024-04-16T07:52:48.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-04-16T07:52:51.873 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.WaitForSafePP (3018 ms) 2024-04-16T07:52:51.873 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-04-16T07:52:54.787 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripPP (2914 ms) 2024-04-16T07:52:54.787 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-04-16T07:52:57.799 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3012 ms) 2024-04-16T07:52:57.799 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-04-16T07:53:00.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3023 ms) 2024-04-16T07:53:00.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-04-16T07:53:03.838 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3016 ms) 2024-04-16T07:53:03.838 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-04-16T07:53:06.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.IsCompletePP (3009 ms) 2024-04-16T07:53:06.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-04-16T07:53:09.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.IsSafePP (3013 ms) 2024-04-16T07:53:09.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-04-16T07:53:12.882 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.ReturnValuePP (3022 ms) 2024-04-16T07:53:12.882 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.FlushPP 2024-04-16T07:53:15.886 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.FlushPP (3004 ms) 2024-04-16T07:53:15.886 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-04-16T07:53:18.905 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3019 ms) 2024-04-16T07:53:18.905 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-04-16T07:53:21.921 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3016 ms) 2024-04-16T07:53:21.921 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-04-16T07:53:24.792 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (2871 ms) 2024-04-16T07:53:24.792 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-04-16T07:53:27.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (3015 ms) 2024-04-16T07:53:27.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-04-16T07:53:30.821 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (3014 ms) 2024-04-16T07:53:30.821 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-04-16T07:53:33.833 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3012 ms) 2024-04-16T07:53:33.833 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-04-16T07:53:36.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.SimpleStatPP (3015 ms) 2024-04-16T07:53:36.849 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-04-16T07:53:39.869 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.OperateMtime (3021 ms) 2024-04-16T07:53:39.869 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-04-16T07:53:42.899 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.OperateMtime2 (3030 ms) 2024-04-16T07:53:42.900 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-04-16T07:53:45.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.StatRemovePP (3011 ms) 2024-04-16T07:53:45.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-04-16T07:53:48.923 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.ExecuteClassPP (3013 ms) 2024-04-16T07:53:48.923 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.OmapPP 2024-04-16T07:53:51.934 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.OmapPP (3011 ms) 2024-04-16T07:53:51.934 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-04-16T07:53:54.801 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.MultiWritePP (2867 ms) 2024-04-16T07:53:54.801 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-04-16T07:53:57.806 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.AioUnlockPP (3005 ms) 2024-04-16T07:53:57.807 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-04-16T07:54:00.836 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (3030 ms) 2024-04-16T07:54:00.836 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-04-16T07:54:03.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RacingRemovePP (3005 ms) 2024-04-16T07:54:03.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-04-16T07:54:06.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (3011 ms) 2024-04-16T07:54:06.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-04-16T07:54:09.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (3009 ms) 2024-04-16T07:54:09.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-04-16T07:54:12.367 INFO:tasks.workunit.client.0.smithi130.stdout:setting pool EIO 2024-04-16T07:54:22.025 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:23.025 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:24.025 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:25.025 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:26.025 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:27.026 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:28.026 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:29.026 INFO:tasks.workunit.client.0.smithi130.stdout:waiting for 474 2024-04-16T07:54:30.026 INFO:tasks.workunit.client.0.smithi130.stdout:max_success 526, min_failed 527 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAio.PoolEIOFlag (20952 ms) 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAio.MultiReads 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:2427: Skipped 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 32 tests from LibRadosAio (129995 ms total) 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 4 tests from LibRadosAioPP 2024-04-16T07:54:30.813 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-04-16T07:54:33.832 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (3018 ms) 2024-04-16T07:54:33.832 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-04-16T07:54:42.816 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (8984 ms) 2024-04-16T07:54:42.816 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-04-16T07:54:57.876 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAioPP.RmXattrPP (15061 ms) 2024-04-16T07:54:57.876 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-04-16T07:55:00.887 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (3011 ms) 2024-04-16T07:55:00.887 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 4 tests from LibRadosAioPP (30074 ms total) 2024-04-16T07:55:00.887 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:00.887 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosIoPP 2024-04-16T07:55:00.887 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-04-16T07:55:03.909 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.XattrListPP (3018 ms) 2024-04-16T07:55:03.909 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosIoPP (3018 ms total) 2024-04-16T07:55:03.909 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:03.909 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 20 tests from LibRadosAioEC 2024-04-16T07:55:03.909 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-04-16T07:55:03.909 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1384: Skipped 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1419: Skipped 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1436: Skipped 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-04-16T07:55:03.910 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1467: Skipped 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1500: Skipped 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1605: Skipped 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1637: Skipped 2024-04-16T07:55:03.911 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1705: Skipped 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1742: Skipped 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1780: Skipped 2024-04-16T07:55:03.912 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1796: Skipped 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1824: Skipped 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (1 ms) 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1859: Skipped 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-04-16T07:55:03.913 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1904: Skipped 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1946: Skipped 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:1977: Skipped 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:2008: Skipped 2024-04-16T07:55:03.914 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:2059: Skipped 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:2089: Skipped 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/aio_cxx.cc:2124: Skipped 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:03.915 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 20 tests from LibRadosAioEC (1 ms total) 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 57 tests from 4 test suites ran. (163088 ms total) 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 36 tests. 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 21 tests, listed below: 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-04-16T07:55:03.916 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-04-16T07:55:03.917 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-04-16T07:55:03.918 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:55:03.918 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:55:03.918 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_io 2024-04-16T07:55:03.926 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:55:03.926 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 24 tests from 2 test suites. 2024-04-16T07:55:03.926 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:55:03.926 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 14 tests from LibRadosIo 2024-04-16T07:55:05.909 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-04-16T07:55:06.299 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.SimpleWrite (389 ms) 2024-04-16T07:55:06.299 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.TooBig 2024-04-16T07:55:06.299 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.TooBig (1 ms) 2024-04-16T07:55:06.299 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-04-16T07:55:06.363 INFO:tasks.workunit.client.0.smithi130.stdout:no timeout :/ 2024-04-16T07:55:06.363 INFO:tasks.workunit.client.0.smithi130.stdout:no timeout :/ 2024-04-16T07:55:06.363 INFO:tasks.workunit.client.0.smithi130.stdout:no timeout :/ 2024-04-16T07:55:06.363 INFO:tasks.workunit.client.0.smithi130.stdout:no timeout :/ 2024-04-16T07:55:06.363 INFO:tasks.workunit.client.0.smithi130.stdout:no timeout :/ 2024-04-16T07:55:06.363 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.ReadTimeout (64 ms) 2024-04-16T07:55:06.363 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-04-16T07:55:06.404 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.RoundTrip (41 ms) 2024-04-16T07:55:06.404 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.Checksum 2024-04-16T07:55:06.419 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.Checksum (15 ms) 2024-04-16T07:55:06.419 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-04-16T07:55:06.450 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (31 ms) 2024-04-16T07:55:06.450 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-04-16T07:55:06.476 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (26 ms) 2024-04-16T07:55:06.476 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-04-16T07:55:06.505 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.AppendRoundTrip (29 ms) 2024-04-16T07:55:06.505 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-04-16T07:55:06.509 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.ZeroLenZero (4 ms) 2024-04-16T07:55:06.509 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.TruncTest 2024-04-16T07:55:06.536 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.TruncTest (27 ms) 2024-04-16T07:55:06.537 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-04-16T07:55:06.562 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.RemoveTest (26 ms) 2024-04-16T07:55:06.562 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-04-16T07:55:06.591 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (29 ms) 2024-04-16T07:55:06.591 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.RmXattr 2024-04-16T07:55:06.669 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.RmXattr (78 ms) 2024-04-16T07:55:06.669 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIo.XattrIter 2024-04-16T07:55:06.705 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIo.XattrIter (36 ms) 2024-04-16T07:55:06.927 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 14 tests from LibRadosIo (796 ms total) 2024-04-16T07:55:06.927 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:06.927 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 10 tests from LibRadosIoEC 2024-04-16T07:55:06.927 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:168: Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-04-16T07:55:06.928 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.929 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (0 ms) 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.930 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:06.931 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:176: Skipped 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 10 tests from LibRadosIoEC (0 ms total) 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 24 tests from 2 test suites ran. (2999 ms total) 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 14 tests. 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 10 tests, listed below: 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-04-16T07:55:06.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:55:06.933 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_io_pp 2024-04-16T07:55:06.945 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:55:06.945 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 38 tests from 2 test suites. 2024-04-16T07:55:06.945 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:55:06.945 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 21 tests from LibRadosIoPP 2024-04-16T07:55:06.945 INFO:tasks.workunit.client.0.smithi130.stdout:seed 34555 2024-04-16T07:55:08.929 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-04-16T07:55:08.930 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-04-16T07:55:08.930 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-04-16T07:55:09.287 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (358 ms) 2024-04-16T07:55:09.287 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-04-16T07:55:09.360 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.ReadOpPP (73 ms) 2024-04-16T07:55:09.360 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-04-16T07:55:09.382 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (21 ms) 2024-04-16T07:55:09.382 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-04-16T07:55:11.946 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (2565 ms) 2024-04-16T07:55:11.946 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-04-16T07:55:11.963 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.RoundTripPP (17 ms) 2024-04-16T07:55:11.963 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-04-16T07:55:11.984 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (21 ms) 2024-04-16T07:55:11.984 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-04-16T07:55:12.001 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.Checksum (18 ms) 2024-04-16T07:55:12.002 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-04-16T07:55:12.018 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (16 ms) 2024-04-16T07:55:12.018 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-04-16T07:55:12.045 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (27 ms) 2024-04-16T07:55:12.045 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-04-16T07:55:12.072 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (27 ms) 2024-04-16T07:55:12.072 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-04-16T07:55:12.088 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (16 ms) 2024-04-16T07:55:12.088 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-04-16T07:55:12.115 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (27 ms) 2024-04-16T07:55:12.116 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-04-16T07:55:12.142 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.TruncTestPP (27 ms) 2024-04-16T07:55:12.142 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-04-16T07:55:12.168 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (26 ms) 2024-04-16T07:55:12.168 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-04-16T07:55:12.198 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (30 ms) 2024-04-16T07:55:12.198 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-04-16T07:55:12.275 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.RmXattrPP (77 ms) 2024-04-16T07:55:12.275 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-04-16T07:55:12.313 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.XattrListPP (38 ms) 2024-04-16T07:55:12.313 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-04-16T07:55:12.341 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.CmpExtPP (28 ms) 2024-04-16T07:55:12.341 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-04-16T07:55:12.359 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (18 ms) 2024-04-16T07:55:12.359 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-04-16T07:55:12.383 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (24 ms) 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 21 tests from LibRadosIoPP (3454 ms total) 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 17 tests from LibRadosIoECPP 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:373: Skipped 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.942 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.943 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-04-16T07:55:12.943 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-04-16T07:55:12.943 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.943 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.943 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.943 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.944 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-04-16T07:55:12.945 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.946 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.947 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-04-16T07:55:12.948 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.949 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (0 ms) 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:381: Skipped 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 17 tests from LibRadosIoECPP (0 ms total) 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 38 tests from 2 test suites ran. (5993 ms total) 2024-04-16T07:55:12.950 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 21 tests. 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 17 tests, listed below: 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-04-16T07:55:12.951 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-04-16T07:55:12.952 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-04-16T07:55:12.952 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-04-16T07:55:12.952 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-04-16T07:55:12.952 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-04-16T07:55:12.952 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:55:12.952 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:55:12.952 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_asio 2024-04-16T07:55:12.971 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.971+0000 7fe8e12356c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:12.977 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.977+0000 7fe8e12356c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:12.977 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.977+0000 7fe8e12356c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:12.978 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 12 tests from 1 test suite. 2024-04-16T07:55:12.978 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:55:12.978 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 12 tests from AsioRados 2024-04-16T07:55:12.978 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.979+0000 7fe8e12356c0 1 Processor -- start 2024-04-16T07:55:12.979 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.979+0000 7fe8e12356c0 1 -- start start 2024-04-16T07:55:12.980 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.980+0000 7fe8e12356c0 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1838690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:55:12.981 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.980+0000 7fe8e12356c0 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x17a3970 con 0x183f900 2024-04-16T07:55:12.981 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.981+0000 7fe8d979b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1838690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:55:12.981 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.981+0000 7fe8d979b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1838690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:35430/0 (socket says 172.21.15.130:35430) 2024-04-16T07:55:12.981 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.981+0000 7fe8d979b640 1 -- 172.21.15.130:0/3511474500 learned_addr learned my addr 172.21.15.130:0/3511474500 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:55:12.982 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.981+0000 7fe8d979b640 1 -- 172.21.15.130:0/3511474500 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x17421e0 con 0x183f900 2024-04-16T07:55:12.982 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.981+0000 7fe8d979b640 1 --2- 172.21.15.130:0/3511474500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1838690 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fcdb489d516ee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:55:12.982 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.981+0000 7fe8d2ffd640 1 -- 172.21.15.130:0/3511474500 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe8c802c980 con 0x183f900 2024-04-16T07:55:12.982 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.981+0000 7fe8d2ffd640 1 -- 172.21.15.130:0/3511474500 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe8c802c390 con 0x183f900 2024-04-16T07:55:12.982 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.982+0000 7fe8d2ffd640 1 -- 172.21.15.130:0/3511474500 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe8c802cb30 con 0x183f900 2024-04-16T07:55:12.983 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.983+0000 7fe8e12356c0 1 -- 172.21.15.130:0/3511474500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 msgr2=0x1838690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:55:12.983 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.983+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/3511474500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1838690 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:12.983 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.983+0000 7fe8e12356c0 1 -- 172.21.15.130:0/3511474500 shutdown_connections 2024-04-16T07:55:12.983 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.983+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/3511474500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1838690 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:12.983 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.983+0000 7fe8e12356c0 1 -- 172.21.15.130:0/3511474500 >> 172.21.15.130:0/3511474500 conn(0x173d2c0 msgr2=0x18cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:55:12.983 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.983+0000 7fe8e12356c0 1 -- 172.21.15.130:0/3511474500 shutdown_connections 2024-04-16T07:55:12.983 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.983+0000 7fe8e12356c0 1 -- 172.21.15.130:0/3511474500 wait complete. 2024-04-16T07:55:12.984 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.984+0000 7fe8e12356c0 1 Processor -- start 2024-04-16T07:55:12.984 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.984+0000 7fe8e12356c0 1 -- start start 2024-04-16T07:55:12.986 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.986+0000 7fe8e12356c0 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1959af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:55:12.986 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.986+0000 7fe8e12356c0 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x17a13f0 con 0x183f900 2024-04-16T07:55:12.987 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.986+0000 7fe8d979b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1959af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:55:12.987 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.986+0000 7fe8d979b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1959af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:35432/0 (socket says 172.21.15.130:35432) 2024-04-16T07:55:12.987 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.986+0000 7fe8d979b640 1 -- 172.21.15.130:0/1948413264 learned_addr learned my addr 172.21.15.130:0/1948413264 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-16T07:55:12.987 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.987+0000 7fe8d979b640 1 -- 172.21.15.130:0/1948413264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1734e30 con 0x183f900 2024-04-16T07:55:12.987 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.987+0000 7fe8d979b640 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1959af0 crc :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:55:12.987 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.987+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe8c803d040 con 0x183f900 2024-04-16T07:55:12.988 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.987+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1734890 con 0x183f900 2024-04-16T07:55:12.988 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.987+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe8c8038b20 con 0x183f900 2024-04-16T07:55:12.988 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.987+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe8c8038dc0 con 0x183f900 2024-04-16T07:55:12.988 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.987+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x18ba280 con 0x183f900 2024-04-16T07:55:12.989 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.988+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fe8c8040040 con 0x183f900 2024-04-16T07:55:12.989 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.989+0000 7fe8d1ffb640 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe8c00371c0 0x7fe8c00396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:55:12.989 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.989+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 2672+0+0 (crc 0 0 0) 0x7fe8c8031040 con 0x183f900 2024-04-16T07:55:12.989 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.989+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x175d6a0 con 0x183f900 2024-04-16T07:55:12.989 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.989+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=235) ==== 24+0+0 (crc 0 0 0) 0x7fe8c8036370 con 0x183f900 2024-04-16T07:55:12.989 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.989+0000 7fe8d8f9a640 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe8c00371c0 0x7fe8c00396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T07:55:12.989 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.989+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) -- 0x17a97d0 con 0x183f900 2024-04-16T07:55:12.990 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:12.990+0000 7fe8d8f9a640 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe8c00371c0 0x7fe8c00396d0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:55:13.938 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:13.938+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v236) ==== 43+0+0 (crc 0 0 0) 0x7fe8c8034290 con 0x183f900 2024-04-16T07:55:13.938 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:13.938+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=236}) -- 0x7fe8c003bb40 con 0x183f900 2024-04-16T07:55:13.941 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:13.941+0000 7fe8d1ffb640 1 -- 172.21.15.130:0/1948413264 <== mon.0 v2:172.21.15.130:3300/0 8 ==== osd_map(236..236 src has 1..236) ==== 651+0+0 (crc 0 0 0) 0x7fe8c8006dc0 con 0x183f900 2024-04-16T07:55:13.941 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:13.941+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/1948413264 >> v2:172.21.15.130:6804/3830191025 conn(0x18d2200 0x196adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T07:55:13.941 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:13.941+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x18ba960 con 0x18d2200 2024-04-16T07:55:13.943 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:13.944+0000 7fe8d9f9c640 1 --2- 172.21.15.130:0/1948413264 >> v2:172.21.15.130:6804/3830191025 conn(0x18d2200 0x196adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-16T07:55:14.046 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:14.046+0000 7fe8d9f9c640 1 --2- 172.21.15.130:0/1948413264 >> v2:172.21.15.130:6804/3830191025 conn(0x18d2200 0x196adf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T07:55:15.215 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.215+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 1 ==== osd_op_reply(2 exist [writefull 0~5] v237'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.215 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-04-16T07:55:15.216 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.215+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1967bc0 con 0x18d2200 2024-04-16T07:55:15.216 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.215+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1968420 con 0x18d2200 2024-04-16T07:55:15.223 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.223+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 2 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.223 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.223+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 3 ==== osd_op_reply(3 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.224 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncReadCallback (8 ms) 2024-04-16T07:55:15.224 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-04-16T07:55:15.224 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.224+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1968800 con 0x18d2200 2024-04-16T07:55:15.224 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.224+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x196b330 con 0x18d2200 2024-04-16T07:55:15.241 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.240+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 4 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.241 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.240+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.241 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncReadFuture (17 ms) 2024-04-16T07:55:15.241 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-04-16T07:55:15.242 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.241+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1967f70 con 0x18d2200 2024-04-16T07:55:15.242 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.241+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x19ac1d0 con 0x18d2200 2024-04-16T07:55:15.252 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.252+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 6 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.253 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.253+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 7 ==== osd_op_reply(7 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.253 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncReadYield (13 ms) 2024-04-16T07:55:15.253 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-04-16T07:55:15.254 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.253+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x196c6a0 con 0x18d2200 2024-04-16T07:55:15.274 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.274+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 8 ==== osd_op_reply(9 exist [write 0~5] v237'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.274 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncWriteCallback (21 ms) 2024-04-16T07:55:15.274 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-04-16T07:55:15.274 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.274+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x196c6a0 con 0x18d2200 2024-04-16T07:55:15.293 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.293+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 9 ==== osd_op_reply(10 exist [write 0~5] v237'3 uv3 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.293 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncWriteFuture (19 ms) 2024-04-16T07:55:15.293 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-04-16T07:55:15.294 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.293+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x198ebd0 con 0x18d2200 2024-04-16T07:55:15.304 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.305+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 10 ==== osd_op_reply(11 exist [write 0~5] v237'4 uv4 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.305 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncWriteYield (12 ms) 2024-04-16T07:55:15.305 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-04-16T07:55:15.305 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.305+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x196b690 con 0x18d2200 2024-04-16T07:55:15.306 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.305+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x196bff0 con 0x18d2200 2024-04-16T07:55:15.312 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.312+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 11 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.312 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.312+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.312 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (7 ms) 2024-04-16T07:55:15.312 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-04-16T07:55:15.313 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.312+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x196bb40 con 0x18d2200 2024-04-16T07:55:15.313 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.312+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x198f280 con 0x18d2200 2024-04-16T07:55:15.320 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.320+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 13 ==== osd_op_reply(14 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.320 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.320+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 14 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.321 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (8 ms) 2024-04-16T07:55:15.321 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-04-16T07:55:15.321 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.320+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x196bb40 con 0x18d2200 2024-04-16T07:55:15.321 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.320+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x196c220 con 0x18d2200 2024-04-16T07:55:15.328 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.328+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 15 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.328 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.328+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.328 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncReadOperationYield (8 ms) 2024-04-16T07:55:15.328 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-04-16T07:55:15.329 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.328+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x196bb40 con 0x18d2200 2024-04-16T07:55:15.339 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.339+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 17 ==== osd_op_reply(18 exist [writefull 0~5] v237'5 uv5 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.340 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (11 ms) 2024-04-16T07:55:15.340 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-04-16T07:55:15.340 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.340+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x196bb40 con 0x18d2200 2024-04-16T07:55:15.350 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.350+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 18 ==== osd_op_reply(19 exist [writefull 0~5] v237'6 uv6 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.351 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (11 ms) 2024-04-16T07:55:15.351 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-04-16T07:55:15.351 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.351+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 --> v2:172.21.15.130:6804/3830191025 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x196bb40 con 0x18d2200 2024-04-16T07:55:15.361 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8d9f9c640 1 -- 172.21.15.130:0/1948413264 <== osd.1 v2:172.21.15.130:6804/3830191025 19 ==== osd_op_reply(20 exist [writefull 0~5] v237'7 uv7 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe8cc00a430 con 0x18d2200 2024-04-16T07:55:15.361 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (11 ms) 2024-04-16T07:55:15.362 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 >> v2:172.21.15.130:6804/3830191025 conn(0x18d2200 msgr2=0x196adf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:55:15.362 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/1948413264 >> v2:172.21.15.130:6804/3830191025 conn(0x18d2200 0x196adf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe8c00371c0 msgr2=0x7fe8c00396d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe8c00371c0 0x7fe8c00396d0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 msgr2=0x1959af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1959af0 crc :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 shutdown_connections 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/1948413264 >> v2:172.21.15.130:6804/3830191025 conn(0x18d2200 0x196adf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.164:6800/4054838717,v1:172.21.15.164:6801/4054838717] conn(0x7fe8c00371c0 0x7fe8c00396d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 --2- 172.21.15.130:0/1948413264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x183f900 0x1959af0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T07:55:15.363 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 >> 172.21.15.130:0/1948413264 conn(0x173d2c0 msgr2=0x18cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T07:55:15.364 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 shutdown_connections 2024-04-16T07:55:15.364 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.361+0000 7fe8e12356c0 1 -- 172.21.15.130:0/1948413264 wait complete. 2024-04-16T07:55:15.364 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 12 tests from AsioRados (146 ms total) 2024-04-16T07:55:15.364 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:15.364 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:55:15.364 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 12 tests from 1 test suite ran. (2383 ms total) 2024-04-16T07:55:15.364 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 12 tests. 2024-04-16T07:55:15.365 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:55:15.365 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:55:15.365 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_list 2024-04-16T07:55:15.393 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.393+0000 7f49b6e299c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:15.393 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.393+0000 7f49b6e299c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:15.394 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:15.393+0000 7f49b6e299c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:15.394 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 11 tests from 3 test suites. 2024-04-16T07:55:15.394 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:55:15.394 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 7 tests from LibRadosList 2024-04-16T07:55:16.955 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosList.ListObjects 2024-04-16T07:55:17.598 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosList.ListObjects (643 ms) 2024-04-16T07:55:17.598 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-04-16T07:55:17.804 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosList.ListObjectsZeroInName (206 ms) 2024-04-16T07:55:17.805 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-04-16T07:55:17.891 INFO:tasks.workunit.client.0.smithi130.stdout:myset foo1,foo2,foo3 2024-04-16T07:55:18.046 INFO:tasks.workunit.client.0.smithi130.stdout:foo1 2024-04-16T07:55:18.047 INFO:tasks.workunit.client.0.smithi130.stdout:foo2 2024-04-16T07:55:18.047 INFO:tasks.workunit.client.0.smithi130.stdout:foo3 2024-04-16T07:55:18.047 INFO:tasks.workunit.client.0.smithi130.stdout:myset foo1,foo4,foo5 2024-04-16T07:55:18.138 INFO:tasks.workunit.client.0.smithi130.stdout:foo4 2024-04-16T07:55:18.139 INFO:tasks.workunit.client.0.smithi130.stdout:foo5 2024-04-16T07:55:18.139 INFO:tasks.workunit.client.0.smithi130.stdout:foo1 2024-04-16T07:55:18.139 INFO:tasks.workunit.client.0.smithi130.stdout:myset foo6,foo7 2024-04-16T07:55:18.231 INFO:tasks.workunit.client.0.smithi130.stdout:foo7 2024-04-16T07:55:18.231 INFO:tasks.workunit.client.0.smithi130.stdout:foo6 2024-04-16T07:55:18.231 INFO:tasks.workunit.client.0.smithi130.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout:ns1:foo4 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout:ns1:foo5 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout:ns2:foo7 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout:ns2:foo6 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout:ns1:foo1 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout::foo1 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout::foo2 2024-04-16T07:55:18.324 INFO:tasks.workunit.client.0.smithi130.stdout::foo3 2024-04-16T07:55:18.505 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosList.ListObjectsNS (701 ms) 2024-04-16T07:55:18.505 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:1 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:10 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:13 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:7 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:14 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:0 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:15 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:11 0 2024-04-16T07:55:18.776 INFO:tasks.workunit.client.0.smithi130.stdout:5 0 2024-04-16T07:55:18.777 INFO:tasks.workunit.client.0.smithi130.stdout:8 0 2024-04-16T07:55:18.777 INFO:tasks.workunit.client.0.smithi130.stdout:6 0 2024-04-16T07:55:18.777 INFO:tasks.workunit.client.0.smithi130.stdout:3 0 2024-04-16T07:55:18.777 INFO:tasks.workunit.client.0.smithi130.stdout:4 0 2024-04-16T07:55:18.777 INFO:tasks.workunit.client.0.smithi130.stdout:12 0 2024-04-16T07:55:18.777 INFO:tasks.workunit.client.0.smithi130.stdout:9 0 2024-04-16T07:55:18.777 INFO:tasks.workunit.client.0.smithi130.stdout:2 0 2024-04-16T07:55:18.869 INFO:tasks.workunit.client.0.smithi130.stdout:have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-04-16T07:55:19.124 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosList.ListObjectsStart (619 ms) 2024-04-16T07:55:19.124 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-04-16T07:55:19.292 INFO:tasks.workunit.client.0.smithi130.stdout:x cursor=MIN 2024-04-16T07:55:19.385 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-04-16T07:55:19.386 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-04-16T07:55:19.387 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-04-16T07:55:19.387 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-04-16T07:55:19.387 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-04-16T07:55:19.387 INFO:tasks.workunit.client.0.smithi130.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-04-16T07:55:19.478 INFO:tasks.workunit.client.0.smithi130.stdout:FIRST> seek to MIN oid=1 2024-04-16T07:55:19.571 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=1 cursor=76:02547ec2:::1:head 2024-04-16T07:55:19.571 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:02547ec2:::1:head 2024-04-16T07:55:19.663 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:02547ec2:::1:head -> 1 2024-04-16T07:55:19.663 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=10 cursor=76:52ea6a34:::10:head 2024-04-16T07:55:19.663 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:52ea6a34:::10:head 2024-04-16T07:55:19.727 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:52ea6a34:::10:head -> 10 2024-04-16T07:55:19.727 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=13 cursor=76:566253c9:::13:head 2024-04-16T07:55:19.727 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:566253c9:::13:head 2024-04-16T07:55:19.790 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:566253c9:::13:head -> 13 2024-04-16T07:55:19.790 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=7 cursor=76:5c6b0b28:::7:head 2024-04-16T07:55:19.790 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:5c6b0b28:::7:head 2024-04-16T07:55:19.849 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:5c6b0b28:::7:head -> 7 2024-04-16T07:55:19.850 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-04-16T07:55:19.850 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:62a1935d:::14:head 2024-04-16T07:55:19.906 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:62a1935d:::14:head -> 14 2024-04-16T07:55:19.906 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=0 cursor=76:6cac518f:::0:head 2024-04-16T07:55:19.906 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:6cac518f:::0:head 2024-04-16T07:55:19.959 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:6cac518f:::0:head -> 0 2024-04-16T07:55:19.960 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=15 cursor=76:863748b0:::15:head 2024-04-16T07:55:19.960 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:863748b0:::15:head 2024-04-16T07:55:20.005 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:863748b0:::15:head -> 15 2024-04-16T07:55:20.005 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=11 cursor=76:89d3ae78:::11:head 2024-04-16T07:55:20.005 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:89d3ae78:::11:head 2024-04-16T07:55:20.048 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:89d3ae78:::11:head -> 11 2024-04-16T07:55:20.048 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=5 cursor=76:b29083e3:::5:head 2024-04-16T07:55:20.048 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:b29083e3:::5:head 2024-04-16T07:55:20.076 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:b29083e3:::5:head -> 5 2024-04-16T07:55:20.076 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=8 cursor=76:bd63b0f1:::8:head 2024-04-16T07:55:20.076 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:bd63b0f1:::8:head 2024-04-16T07:55:20.101 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:bd63b0f1:::8:head -> 8 2024-04-16T07:55:20.101 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=6 cursor=76:c4fdafeb:::6:head 2024-04-16T07:55:20.101 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:c4fdafeb:::6:head 2024-04-16T07:55:20.123 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:c4fdafeb:::6:head -> 6 2024-04-16T07:55:20.123 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=3 cursor=76:cfc208b3:::3:head 2024-04-16T07:55:20.123 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:cfc208b3:::3:head 2024-04-16T07:55:20.142 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:cfc208b3:::3:head -> 3 2024-04-16T07:55:20.142 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=4 cursor=76:d83876eb:::4:head 2024-04-16T07:55:20.143 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:d83876eb:::4:head 2024-04-16T07:55:20.155 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:d83876eb:::4:head -> 4 2024-04-16T07:55:20.156 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=12 cursor=76:de5d7c5f:::12:head 2024-04-16T07:55:20.156 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:de5d7c5f:::12:head 2024-04-16T07:55:20.170 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:de5d7c5f:::12:head -> 12 2024-04-16T07:55:20.170 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=9 cursor=76:e960b815:::9:head 2024-04-16T07:55:20.170 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:e960b815:::9:head 2024-04-16T07:55:20.178 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:e960b815:::9:head -> 9 2024-04-16T07:55:20.178 INFO:tasks.workunit.client.0.smithi130.stdout:: oid=2 cursor=76:f905c69b:::2:head 2024-04-16T07:55:20.178 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:f905c69b:::2:head 2024-04-16T07:55:20.181 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:f905c69b:::2:head -> 2 2024-04-16T07:55:20.181 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:89d3ae78:::11:head 2024-04-16T07:55:20.181 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:89d3ae78:::11:head expected=76:89d3ae78:::11:head 2024-04-16T07:55:20.223 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:89d3ae78:::11:head -> 11 2024-04-16T07:55:20.223 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=11 expected=11 2024-04-16T07:55:20.223 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:02547ec2:::1:head 2024-04-16T07:55:20.223 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:02547ec2:::1:head expected=76:02547ec2:::1:head 2024-04-16T07:55:20.317 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:02547ec2:::1:head -> 1 2024-04-16T07:55:20.317 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=1 expected=1 2024-04-16T07:55:20.317 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:de5d7c5f:::12:head 2024-04-16T07:55:20.317 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:de5d7c5f:::12:head expected=76:de5d7c5f:::12:head 2024-04-16T07:55:20.330 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:de5d7c5f:::12:head -> 12 2024-04-16T07:55:20.330 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=12 expected=12 2024-04-16T07:55:20.330 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:d83876eb:::4:head 2024-04-16T07:55:20.330 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:d83876eb:::4:head expected=76:d83876eb:::4:head 2024-04-16T07:55:20.343 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:d83876eb:::4:head -> 4 2024-04-16T07:55:20.344 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=4 expected=4 2024-04-16T07:55:20.344 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:cfc208b3:::3:head 2024-04-16T07:55:20.344 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:cfc208b3:::3:head expected=76:cfc208b3:::3:head 2024-04-16T07:55:20.362 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:cfc208b3:::3:head -> 3 2024-04-16T07:55:20.362 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=3 expected=3 2024-04-16T07:55:20.363 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:f905c69b:::2:head 2024-04-16T07:55:20.363 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:f905c69b:::2:head expected=76:f905c69b:::2:head 2024-04-16T07:55:20.365 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:f905c69b:::2:head -> 2 2024-04-16T07:55:20.365 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=2 expected=2 2024-04-16T07:55:20.365 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:e960b815:::9:head 2024-04-16T07:55:20.365 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:e960b815:::9:head expected=76:e960b815:::9:head 2024-04-16T07:55:20.374 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:e960b815:::9:head -> 9 2024-04-16T07:55:20.374 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=9 expected=9 2024-04-16T07:55:20.374 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:c4fdafeb:::6:head 2024-04-16T07:55:20.374 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:c4fdafeb:::6:head expected=76:c4fdafeb:::6:head 2024-04-16T07:55:20.396 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:c4fdafeb:::6:head -> 6 2024-04-16T07:55:20.397 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=6 expected=6 2024-04-16T07:55:20.397 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:bd63b0f1:::8:head 2024-04-16T07:55:20.397 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:bd63b0f1:::8:head expected=76:bd63b0f1:::8:head 2024-04-16T07:55:20.421 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:bd63b0f1:::8:head -> 8 2024-04-16T07:55:20.421 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=8 expected=8 2024-04-16T07:55:20.421 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:b29083e3:::5:head 2024-04-16T07:55:20.421 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:b29083e3:::5:head expected=76:b29083e3:::5:head 2024-04-16T07:55:20.449 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:b29083e3:::5:head -> 5 2024-04-16T07:55:20.449 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=5 expected=5 2024-04-16T07:55:20.449 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:863748b0:::15:head 2024-04-16T07:55:20.449 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:863748b0:::15:head expected=76:863748b0:::15:head 2024-04-16T07:55:20.503 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:863748b0:::15:head -> 15 2024-04-16T07:55:20.503 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=15 expected=15 2024-04-16T07:55:20.504 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:6cac518f:::0:head 2024-04-16T07:55:20.504 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:6cac518f:::0:head expected=76:6cac518f:::0:head 2024-04-16T07:55:20.558 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:6cac518f:::0:head -> 0 2024-04-16T07:55:20.558 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=0 expected=0 2024-04-16T07:55:20.558 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:62a1935d:::14:head 2024-04-16T07:55:20.558 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:62a1935d:::14:head expected=76:62a1935d:::14:head 2024-04-16T07:55:20.616 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:62a1935d:::14:head -> 14 2024-04-16T07:55:20.616 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=14 expected=14 2024-04-16T07:55:20.616 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:5c6b0b28:::7:head 2024-04-16T07:55:20.616 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:5c6b0b28:::7:head expected=76:5c6b0b28:::7:head 2024-04-16T07:55:20.676 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:5c6b0b28:::7:head -> 7 2024-04-16T07:55:20.676 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=7 expected=7 2024-04-16T07:55:20.676 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:566253c9:::13:head 2024-04-16T07:55:20.676 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:566253c9:::13:head expected=76:566253c9:::13:head 2024-04-16T07:55:20.738 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:566253c9:::13:head -> 13 2024-04-16T07:55:20.738 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=13 expected=13 2024-04-16T07:55:20.738 INFO:tasks.workunit.client.0.smithi130.stdout:: seek to 76:52ea6a34:::10:head 2024-04-16T07:55:20.739 INFO:tasks.workunit.client.0.smithi130.stdout:: cursor()=76:52ea6a34:::10:head expected=76:52ea6a34:::10:head 2024-04-16T07:55:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:> 76:52ea6a34:::10:head -> 10 2024-04-16T07:55:20.802 INFO:tasks.workunit.client.0.smithi130.stdout:: entry=10 expected=10 2024-04-16T07:55:21.058 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosList.ListObjectsCursor (1934 ms) 2024-04-16T07:55:21.058 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-04-16T07:55:21.584 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosList.EnumerateObjects (526 ms) 2024-04-16T07:55:21.584 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-04-16T07:55:21.754 INFO:tasks.workunit.client.0.smithi130.stdout:split 0/5 -> MIN 76:33333333::::head 2024-04-16T07:55:21.775 INFO:tasks.workunit.client.0.smithi130.stdout:split 1/5 -> 76:33333333::::head 76:66666666::::head 2024-04-16T07:55:21.797 INFO:tasks.workunit.client.0.smithi130.stdout:split 2/5 -> 76:66666666::::head 76:99999999::::head 2024-04-16T07:55:21.821 INFO:tasks.workunit.client.0.smithi130.stdout:split 3/5 -> 76:99999999::::head 76:cccccccc::::head 2024-04-16T07:55:21.841 INFO:tasks.workunit.client.0.smithi130.stdout:split 4/5 -> 76:cccccccc::::head MAX 2024-04-16T07:55:22.125 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosList.EnumerateObjectsSplit (541 ms) 2024-04-16T07:55:22.292 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 7 tests from LibRadosList (5170 ms total) 2024-04-16T07:55:22.292 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:22.292 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 3 tests from LibRadosListEC 2024-04-16T07:55:22.292 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:70: Skipped 2024-04-16T07:55:22.292 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.292 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-04-16T07:55:22.292 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:84: Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:96: Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:84: Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:96: Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:84: Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:96: Skipped 2024-04-16T07:55:22.293 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.294 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-04-16T07:55:22.294 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:78: Skipped 2024-04-16T07:55:22.294 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:22.294 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-04-16T07:55:22.294 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:22.294 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosListNP 2024-04-16T07:55:22.294 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-04-16T07:55:24.813 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosListNP.ListObjectsError (2521 ms) 2024-04-16T07:55:24.813 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosListNP (2521 ms total) 2024-04-16T07:55:24.813 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:24.813 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:55:24.813 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 11 tests from 3 test suites ran. (9420 ms total) 2024-04-16T07:55:24.814 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 8 tests. 2024-04-16T07:55:24.814 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 3 tests, listed below: 2024-04-16T07:55:24.814 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-04-16T07:55:24.814 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-04-16T07:55:24.814 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-04-16T07:55:24.816 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:55:24.816 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:55:24.816 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_lock 2024-04-16T07:55:24.833 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:55:24.833 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 16 tests from 2 test suites. 2024-04-16T07:55:24.833 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:55:24.833 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLock 2024-04-16T07:55:26.820 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.LockExclusive 2024-04-16T07:55:27.243 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.LockExclusive (423 ms) 2024-04-16T07:55:27.243 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.LockShared 2024-04-16T07:55:27.283 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.LockShared (40 ms) 2024-04-16T07:55:27.284 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.LockExclusiveDur 2024-04-16T07:55:28.319 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.LockExclusiveDur (1036 ms) 2024-04-16T07:55:28.319 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.LockSharedDur 2024-04-16T07:55:29.471 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.LockSharedDur (1152 ms) 2024-04-16T07:55:29.472 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.LockMayRenew 2024-04-16T07:55:29.506 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.LockMayRenew (34 ms) 2024-04-16T07:55:29.506 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.Unlock 2024-04-16T07:55:29.550 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.Unlock (44 ms) 2024-04-16T07:55:29.550 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.ListLockers 2024-04-16T07:55:29.602 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.ListLockers (52 ms) 2024-04-16T07:55:29.602 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLock.BreakLock 2024-04-16T07:55:29.631 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLock.BreakLock (28 ms) 2024-04-16T07:55:29.820 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLock (2811 ms total) 2024-04-16T07:55:29.820 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:29.820 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLockEC 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:168: Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.LockExclusive 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive (0 ms) 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.LockShared 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.821 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockShared (0 ms) 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.LockExclusiveDur 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur (0 ms) 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.LockSharedDur 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur (0 ms) 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.LockMayRenew 2024-04-16T07:55:29.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew (0 ms) 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.Unlock 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.Unlock (0 ms) 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.ListLockers 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers (0 ms) 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockEC.BreakLock 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:182: Skipped 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:196: Skipped 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock (0 ms) 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/TestCase.cc:176: Skipped 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLockEC (0 ms total) 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 16 tests from 2 test suites ran. (4985 ms total) 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 8 tests. 2024-04-16T07:55:29.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 8 tests, listed below: 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockShared 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.Unlock 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock 2024-04-16T07:55:29.825 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:55:29.826 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:55:29.826 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_lock_pp 2024-04-16T07:55:29.838 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:55:29.839 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 16 tests from 2 test suites. 2024-04-16T07:55:29.839 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:55:29.839 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLockPP 2024-04-16T07:55:29.839 INFO:tasks.workunit.client.0.smithi130.stdout:seed 34670 2024-04-16T07:55:31.840 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.LockExclusivePP 2024-04-16T07:55:32.177 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.LockExclusivePP (337 ms) 2024-04-16T07:55:32.177 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.LockSharedPP 2024-04-16T07:55:32.202 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.LockSharedPP (25 ms) 2024-04-16T07:55:32.202 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-04-16T07:55:33.241 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.LockExclusiveDurPP (1039 ms) 2024-04-16T07:55:33.242 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.LockSharedDurPP 2024-04-16T07:55:34.284 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.LockSharedDurPP (1043 ms) 2024-04-16T07:55:34.284 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.LockMayRenewPP 2024-04-16T07:55:34.324 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.LockMayRenewPP (40 ms) 2024-04-16T07:55:34.324 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.UnlockPP 2024-04-16T07:55:34.368 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.UnlockPP (44 ms) 2024-04-16T07:55:34.368 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.ListLockersPP 2024-04-16T07:55:34.421 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.ListLockersPP (53 ms) 2024-04-16T07:55:34.422 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockPP.BreakLockPP 2024-04-16T07:55:34.452 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosLockPP.BreakLockPP (31 ms) 2024-04-16T07:55:34.821 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLockPP (2612 ms total) 2024-04-16T07:55:34.821 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:34.821 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLockECPP 2024-04-16T07:55:34.821 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:373: Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.LockExclusivePP 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP (0 ms) 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.LockSharedPP 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.822 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP (0 ms) 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP (1 ms) 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP (0 ms) 2024-04-16T07:55:34.823 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP (0 ms) 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.UnlockPP 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP (0 ms) 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.ListLockersPP 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.824 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP (0 ms) 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosLockECPP.BreakLockPP 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP (0 ms) 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:381: Skipped 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosLockECPP (1 ms total) 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 16 tests from 2 test suites ran. (4982 ms total) 2024-04-16T07:55:34.825 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 8 tests. 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 8 tests, listed below: 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP 2024-04-16T07:55:34.826 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP 2024-04-16T07:55:34.827 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:55:34.827 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:55:34.827 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_misc 2024-04-16T07:55:34.855 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:34.855+0000 7f8f1288d6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:34.855+0000 7f8f1288d6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:34.855+0000 7f8f1288d6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 12 tests from 4 test suites. 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscVersion 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscVersion.Version 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscVersion.Version (0 ms) 2024-04-16T07:55:34.856 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-16T07:55:34.857 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:34.857 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosMiscConnectFailure 2024-04-16T07:55:34.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-04-16T07:55:34.919 INFO:tasks.workunit.client.0.smithi130.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2024-04-16T07:55:34.920 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:34.920+0000 7f8f1288d6c0 -1 failed for service _ceph-mon._tcp 2024-04-16T07:55:34.920 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:55:34.920+0000 7f8f1288d6c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-04-16T07:55:34.920 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectFailure (63 ms) 2024-04-16T07:55:34.920 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-04-16T07:55:36.926 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2007 ms) 2024-04-16T07:55:36.927 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosMiscConnectFailure (2070 ms total) 2024-04-16T07:55:36.927 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:36.927 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscPool 2024-04-16T07:55:36.927 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPool.PoolCreationRace 2024-04-16T07:55:39.831 INFO:tasks.workunit.client.0.smithi130.stdout:started 0x7f8ed8005de0 2024-04-16T07:55:40.831 INFO:tasks.workunit.client.0.smithi130.stdout:started 0x7f8eec005de0 2024-04-16T07:55:40.831 INFO:tasks.workunit.client.0.smithi130.stdout: started 2 aios 2024-04-16T07:55:40.831 INFO:tasks.workunit.client.0.smithi130.stdout:waiting 0x7f8ed8005de0 2024-04-16T07:55:40.831 INFO:tasks.workunit.client.0.smithi130.stdout:waiting 0x7f8eec005de0 2024-04-16T07:55:40.836 INFO:tasks.workunit.client.0.smithi130.stdout:done. 2024-04-16T07:55:42.845 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPool.PoolCreationRace (5919 ms) 2024-04-16T07:55:42.845 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscPool (5919 ms total) 2024-04-16T07:55:42.845 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:55:42.845 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosMisc 2024-04-16T07:55:44.820 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.ClusterFSID 2024-04-16T07:55:44.820 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-04-16T07:55:44.820 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.Exec 2024-04-16T07:55:45.126 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.Exec (307 ms) 2024-04-16T07:55:45.126 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.WriteSame 2024-04-16T07:55:45.198 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.WriteSame (71 ms) 2024-04-16T07:55:45.198 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.CmpExt 2024-04-16T07:55:45.218 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.CmpExt (21 ms) 2024-04-16T07:55:45.218 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.Applications 2024-04-16T07:55:49.882 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.Applications (4664 ms) 2024-04-16T07:55:49.882 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.MinCompatOSD 2024-04-16T07:55:49.883 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-04-16T07:55:49.883 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.MinCompatClient 2024-04-16T07:55:49.883 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-04-16T07:55:49.883 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMisc.ShutdownRace 2024-04-16T07:56:15.582 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMisc.ShutdownRace (25699 ms) 2024-04-16T07:56:15.834 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 8 tests from LibRadosMisc (30762 ms total) 2024-04-16T07:56:15.834 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:15.834 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:56:15.834 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 12 tests from 4 test suites ran. (40978 ms total) 2024-04-16T07:56:15.835 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 12 tests. 2024-04-16T07:56:15.847 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:56:15.847 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:56:15.847 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_misc_pp 2024-04-16T07:56:15.880 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:56:15.880+0000 7f370a832840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:56:15.881 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:56:15.880+0000 7f370a832840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:56:15.881 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:56:15.880+0000 7f370a832840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:56:15.881 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 31 tests from 7 test suites. 2024-04-16T07:56:15.881 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:56:15.881 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscVersion 2024-04-16T07:56:15.881 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscVersion.VersionPP 2024-04-16T07:56:15.882 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-04-16T07:56:15.882 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-16T07:56:15.882 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:15.882 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 22 tests from LibRadosMiscPP 2024-04-16T07:56:15.882 INFO:tasks.workunit.client.0.smithi130.stdout:seed 96336 2024-04-16T07:56:17.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-04-16T07:56:17.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-04-16T07:56:17.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.LongNamePP 2024-04-16T07:56:18.370 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.LongNamePP (523 ms) 2024-04-16T07:56:18.370 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.LongLocatorPP 2024-04-16T07:56:18.522 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.LongLocatorPP (152 ms) 2024-04-16T07:56:18.522 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.LongNSpacePP 2024-04-16T07:56:18.575 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.LongNSpacePP (53 ms) 2024-04-16T07:56:18.576 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-04-16T07:56:18.620 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.LongAttrNamePP (45 ms) 2024-04-16T07:56:18.620 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.ExecPP 2024-04-16T07:56:18.636 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.ExecPP (16 ms) 2024-04-16T07:56:18.636 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.BadFlagsPP 2024-04-16T07:56:18.650 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.BadFlagsPP (15 ms) 2024-04-16T07:56:18.651 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.Operate1PP 2024-04-16T07:56:18.676 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.Operate1PP (25 ms) 2024-04-16T07:56:18.676 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.Operate2PP 2024-04-16T07:56:18.693 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.Operate2PP (17 ms) 2024-04-16T07:56:18.693 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.BigObjectPP 2024-04-16T07:56:18.741 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.BigObjectPP (48 ms) 2024-04-16T07:56:18.741 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.AioOperatePP 2024-04-16T07:56:18.757 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.AioOperatePP (16 ms) 2024-04-16T07:56:18.757 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.AssertExistsPP 2024-04-16T07:56:18.781 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.AssertExistsPP (24 ms) 2024-04-16T07:56:18.781 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.AssertVersionPP 2024-04-16T07:56:18.832 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.AssertVersionPP (51 ms) 2024-04-16T07:56:18.832 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.BigAttrPP 2024-04-16T07:56:18.843 INFO:tasks.workunit.client.0.smithi130.stdout:osd_max_attr_size = 0 2024-04-16T07:56:18.843 INFO:tasks.workunit.client.0.smithi130.stdout:osd_max_attr_size == 0; skipping test 2024-04-16T07:56:37.033 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.BigAttrPP (18200 ms) 2024-04-16T07:56:37.033 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.CopyPP 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/misc_cxx.cc:386: Skipped 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP (1 ms) 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.CopyScrubPP 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/misc_cxx.cc:549: Skipped 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP (0 ms) 2024-04-16T07:56:37.034 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.WriteSamePP 2024-04-16T07:56:37.080 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.WriteSamePP (47 ms) 2024-04-16T07:56:37.080 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.CmpExtPP 2024-04-16T07:56:37.102 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.CmpExtPP (22 ms) 2024-04-16T07:56:37.102 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.Applications 2024-04-16T07:56:41.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.Applications (4830 ms) 2024-04-16T07:56:41.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.MinCompatOSD 2024-04-16T07:56:41.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-04-16T07:56:41.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.MinCompatClient 2024-04-16T07:56:41.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-04-16T07:56:41.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscPP.Conf 2024-04-16T07:56:41.932 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosMiscPP.Conf (0 ms) 2024-04-16T07:56:42.936 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 22 tests from LibRadosMiscPP (24086 ms total) 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosTwoPoolsECPP 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/misc_cxx.cc:476: Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/misc_cxx.cc:498: Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/misc_cxx.cc:504: Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom (0 ms) 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/misc_cxx.cc:491: Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosTwoPoolsECPP (0 ms total) 2024-04-16T07:56:42.937 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:42.938 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-04-16T07:56:44.827 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosChecksum/0.Subset 2024-04-16T07:56:45.217 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosChecksum/0.Subset (391 ms) 2024-04-16T07:56:45.218 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosChecksum/0.Chunked 2024-04-16T07:56:45.240 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosChecksum/0.Chunked (22 ms) 2024-04-16T07:56:45.955 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosChecksum/0 (413 ms total) 2024-04-16T07:56:45.955 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:45.955 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-04-16T07:56:47.968 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosChecksum/1.Subset 2024-04-16T07:56:48.342 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosChecksum/1.Subset (374 ms) 2024-04-16T07:56:48.342 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosChecksum/1.Chunked 2024-04-16T07:56:48.385 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosChecksum/1.Chunked (43 ms) 2024-04-16T07:56:48.974 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosChecksum/1 (417 ms total) 2024-04-16T07:56:48.975 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:48.975 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-04-16T07:56:50.974 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosChecksum/2.Subset 2024-04-16T07:56:51.342 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosChecksum/2.Subset (369 ms) 2024-04-16T07:56:51.342 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosChecksum/2.Chunked 2024-04-16T07:56:51.384 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosChecksum/2.Chunked (42 ms) 2024-04-16T07:56:51.991 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 2 tests from LibRadosChecksum/2 (411 ms total) 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscECPP 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:373: Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange (0 ms) 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:381: Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 1 test from LibRadosMiscECPP (0 ms total) 2024-04-16T07:56:51.992 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 31 tests from 7 test suites ran. (36110 ms total) 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 27 tests. 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 4 tests, listed below: 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom 2024-04-16T07:56:51.993 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange 2024-04-16T07:56:51.994 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:56:51.994 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:56:51.994 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_tier_pp 2024-04-16T07:56:52.026 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:56:52.026+0000 7f6247a589c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:56:52.026 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:56:52.026+0000 7f6247a589c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:56:52.027 INFO:tasks.workunit.client.0.smithi130.stderr:2024-04-16T07:56:52.026+0000 7f6247a589c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T07:56:52.027 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 77 tests from 4 test suites. 2024-04-16T07:56:52.027 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:56:52.027 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 3 tests from LibRadosTierPP 2024-04-16T07:56:52.027 INFO:tasks.workunit.client.0.smithi130.stdout:seed 96408 2024-04-16T07:56:54.004 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTierPP.Dirty 2024-04-16T07:56:54.004 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:287: Skipped 2024-04-16T07:56:54.004 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierPP.Dirty (0 ms) 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTierPP.FlushWriteRaces 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:2182: Skipped 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces (0 ms) 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTierPP.HitSetNone 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:2561: Skipped 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:54.005 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone (0 ms) 2024-04-16T07:56:54.830 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 3 tests from LibRadosTierPP (0 ms total) 2024-04-16T07:56:54.830 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:54.830 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 48 tests from LibRadosTwoPoolsPP 2024-04-16T07:56:56.839 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.Overlay 2024-04-16T07:56:56.839 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay (0 ms) 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.Promote 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote (0 ms) 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.840 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap (0 ms) 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub (0 ms) 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.841 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (0 ms) 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout (0 ms) 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.842 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (0 ms) 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.Evict 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict (0 ms) 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.843 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap (1 ms) 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 (0 ms) 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap (0 ms) 2024-04-16T07:56:56.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-04-16T07:56:56.845 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.845 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.845 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.845 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.845 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (0 ms) 2024-04-16T07:56:56.845 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-04-16T07:56:56.845 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush (0 ms) 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.Flush 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush (0 ms) 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.846 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap (0 ms) 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces (0 ms) 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.847 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace (0 ms) 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead (0 ms) 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite (0 ms) 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim (0 ms) 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead (0 ms) 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-04-16T07:56:56.849 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead (0 ms) 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.CachePin 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin (0 ms) 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead (0 ms) 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead (0 ms) 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead (0 ms) 2024-04-16T07:56:56.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset (0 ms) 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead (0 ms) 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount (0 ms) 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (0 ms) 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (0 ms) 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (0 ms) 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (0 ms) 2024-04-16T07:56:56.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount (0 ms) 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict (0 ms) 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote (0 ms) 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (0 ms) 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-04-16T07:56:56.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead (0 ms) 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap (0 ms) 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount (0 ms) 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush (0 ms) 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (0 ms) 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback (0 ms) 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (0 ms) 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-04-16T07:56:56.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback (0 ms) 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError (0 ms) 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.861 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn (0 ms) 2024-04-16T07:56:56.861 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-04-16T07:56:56.861 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:223: Skipped 2024-04-16T07:56:56.861 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.861 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:233: Skipped 2024-04-16T07:56:56.861 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:56.861 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (0 ms) 2024-04-16T07:56:57.849 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 48 tests from LibRadosTwoPoolsPP (1 ms total) 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 4 tests from LibRadosTierECPP 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:373: Skipped 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTierECPP.Dirty 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty (0 ms) 2024-04-16T07:56:57.850 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces (0 ms) 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTierECPP.CallForcesPromote 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote (0 ms) 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTierECPP.HitSetNone 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-16T07:56:57.851 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone (0 ms) 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/testcase_cxx.cc:381: Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 4 tests from LibRadosTierECPP (0 ms total) 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6267: Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.852 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay (0 ms) 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.Promote 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote (0 ms) 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap (0 ms) 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-04-16T07:56:57.853 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (0 ms) 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout (0 ms) 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.854 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict (0 ms) 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap (0 ms) 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.855 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush (0 ms) 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush (0 ms) 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush (0 ms) 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.856 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap (0 ms) 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (0 ms) 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.857 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace (0 ms) 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead (0 ms) 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim (0 ms) 2024-04-16T07:56:57.858 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (0 ms) 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead (0 ms) 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.859 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin (0 ms) 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead (0 ms) 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.860 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead (0 ms) 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead (0 ms) 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6278: Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6288: Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier (0 ms) 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/tier_cxx.cc:6272: Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP (0 ms total) 2024-04-16T07:56:57.861 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 77 tests from 4 test suites ran. (5811 ms total) 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 0 tests. 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 77 tests, listed below: 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierPP.Dirty 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout 2024-04-16T07:56:57.862 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead 2024-04-16T07:56:57.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead 2024-04-16T07:56:57.864 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote 2024-04-16T07:56:57.865 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead 2024-04-16T07:56:57.866 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_pool 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:56:57.867 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-16T07:56:57.868 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:56:57.868 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 9 tests from LibRadosPools 2024-04-16T07:56:57.868 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolList 2024-04-16T07:57:00.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolList (2985 ms) 2024-04-16T07:57:00.848 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolLookup 2024-04-16T07:57:03.872 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolLookup (3024 ms) 2024-04-16T07:57:03.872 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolLookup2 2024-04-16T07:57:06.836 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolLookup2 (2964 ms) 2024-04-16T07:57:06.837 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-04-16T07:57:09.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolLookupOtherInstance (3016 ms) 2024-04-16T07:57:09.852 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-04-16T07:57:12.871 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3019 ms) 2024-04-16T07:57:12.871 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolDelete 2024-04-16T07:57:17.844 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolDelete (4973 ms) 2024-04-16T07:57:17.845 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolCreateDelete 2024-04-16T07:57:22.870 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolCreateDelete (5026 ms) 2024-04-16T07:57:22.870 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosPools.PoolCreateWithCrushRule (4971 ms) 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosPools.PoolGetBaseTier 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3109-g30b0aa00/rpm/el9/BUILD/ceph-19.0.0-3109-g30b0aa00/src/test/librados/pool.cc:139: Skipped 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout:Not supported by crimson yet. Skipped 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier (0 ms) 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 9 tests from LibRadosPools (29978 ms total) 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:57:27.841 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment tear-down 2024-04-16T07:57:27.842 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] 9 tests from 1 test suite ran. (29978 ms total) 2024-04-16T07:57:27.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ PASSED ] 8 tests. 2024-04-16T07:57:27.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] 1 test, listed below: 2024-04-16T07:57:27.842 INFO:tasks.workunit.client.0.smithi130.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier 2024-04-16T07:57:27.843 INFO:tasks.workunit.client.0.smithi130.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-16T07:57:27.843 INFO:tasks.workunit.client.0.smithi130.stderr:+ '[' 0 -eq 1 ']' 2024-04-16T07:57:27.844 INFO:tasks.workunit.client.0.smithi130.stderr:+ ceph_test_rados_api_snapshots 2024-04-16T07:57:27.865 INFO:tasks.workunit.client.0.smithi130.stdout:Running main() from gmock_main.cc 2024-04-16T07:57:27.865 INFO:tasks.workunit.client.0.smithi130.stdout:[==========] Running 13 tests from 4 test suites. 2024-04-16T07:57:27.865 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] Global test environment set-up. 2024-04-16T07:57:27.865 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 4 tests from LibRadosSnapshots 2024-04-16T07:57:29.854 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosSnapshots.SnapList 2024-04-16T07:57:31.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosSnapshots.SnapList (2008 ms) 2024-04-16T07:57:31.863 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosSnapshots.SnapRemove 2024-04-16T07:57:33.878 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosSnapshots.SnapRemove (2016 ms) 2024-04-16T07:57:33.878 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosSnapshots.Rollback 2024-04-16T07:57:35.885 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosSnapshots.Rollback (2007 ms) 2024-04-16T07:57:35.885 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosSnapshots.SnapGetName 2024-04-16T07:57:37.891 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosSnapshots.SnapGetName (2005 ms) 2024-04-16T07:57:38.898 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 4 tests from LibRadosSnapshots (8036 ms total) 2024-04-16T07:57:38.899 INFO:tasks.workunit.client.0.smithi130.stdout: 2024-04-16T07:57:38.899 INFO:tasks.workunit.client.0.smithi130.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-04-16T07:57:40.905 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-04-16T07:57:44.991 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Snap (4086 ms) 2024-04-16T07:57:44.991 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-04-16T07:57:49.116 INFO:tasks.workunit.client.0.smithi130.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Rollback (4125 ms) 2024-04-16T07:57:49.116 INFO:tasks.workunit.client.0.smithi130.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-04-16T07:57:57.112 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-16T07:58:02.616 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~6s 2024-04-16T07:58:08.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~11s 2024-04-16T07:58:13.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~17s 2024-04-16T07:58:19.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~22s 2024-04-16T07:58:24.534 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-16T07:58:24.534 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~28s 2024-04-16T07:58:29.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2024-04-16T07:58:29.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~33s 2024-04-16T07:58:35.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~11s 2024-04-16T07:58:35.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~38s 2024-04-16T07:58:40.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~16s 2024-04-16T07:58:40.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~44s 2024-04-16T07:58:46.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~22s 2024-04-16T07:58:46.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~49s 2024-04-16T07:58:51.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~27s 2024-04-16T07:58:51.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~55s 2024-04-16T07:58:56.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~32s 2024-04-16T07:58:56.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~60s 2024-04-16T07:59:02.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~38s 2024-04-16T07:59:02.364 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~65s 2024-04-16T07:59:04.198 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:59:04.198+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T07:59:04.198 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T07:59:04.198+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T07:59:07.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~43s 2024-04-16T07:59:07.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~71s 2024-04-16T07:59:13.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~49s 2024-04-16T07:59:13.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~76s 2024-04-16T07:59:18.575 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~54s 2024-04-16T07:59:18.575 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~82s 2024-04-16T07:59:23.979 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~59s 2024-04-16T07:59:23.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~87s 2024-04-16T07:59:29.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~65s 2024-04-16T07:59:29.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~92s 2024-04-16T07:59:34.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2024-04-16T07:59:34.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~98s 2024-04-16T07:59:40.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~76s 2024-04-16T07:59:40.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~103s 2024-04-16T07:59:45.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~81s 2024-04-16T07:59:45.596 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~109s 2024-04-16T07:59:50.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~86s 2024-04-16T07:59:51.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~114s 2024-04-16T07:59:56.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~92s 2024-04-16T07:59:56.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~119s 2024-04-16T08:00:01.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~97s 2024-04-16T08:00:01.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~125s 2024-04-16T08:00:04.874 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:00:04.875+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T08:00:04.875 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:00:04.875+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T08:00:07.212 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~103s 2024-04-16T08:00:07.212 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~130s 2024-04-16T08:00:12.615 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~108s 2024-04-16T08:00:12.615 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~136s 2024-04-16T08:00:18.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~113s 2024-04-16T08:00:18.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~141s 2024-04-16T08:00:23.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~119s 2024-04-16T08:00:23.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~146s 2024-04-16T08:00:28.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~124s 2024-04-16T08:00:28.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~152s 2024-04-16T08:00:34.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~130s 2024-04-16T08:00:34.231 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~157s 2024-04-16T08:00:39.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~135s 2024-04-16T08:00:39.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~163s 2024-04-16T08:00:45.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~141s 2024-04-16T08:00:45.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~168s 2024-04-16T08:00:50.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~146s 2024-04-16T08:00:50.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~173s 2024-04-16T08:00:55.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~151s 2024-04-16T08:00:55.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~179s 2024-04-16T08:01:01.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~157s 2024-04-16T08:01:01.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~184s 2024-04-16T08:01:04.879 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:01:04.878+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T08:01:04.879 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:01:04.878+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T08:01:06.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~162s 2024-04-16T08:01:06.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~190s 2024-04-16T08:01:12.058 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~168s 2024-04-16T08:01:12.058 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~195s 2024-04-16T08:01:17.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~173s 2024-04-16T08:01:17.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~200s 2024-04-16T08:01:22.866 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~178s 2024-04-16T08:01:22.866 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~206s 2024-04-16T08:01:28.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~184s 2024-04-16T08:01:28.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~211s 2024-04-16T08:01:33.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~189s 2024-04-16T08:01:33.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~217s 2024-04-16T08:01:39.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~195s 2024-04-16T08:01:39.079 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~222s 2024-04-16T08:01:44.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~200s 2024-04-16T08:01:44.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~227s 2024-04-16T08:01:49.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~205s 2024-04-16T08:01:49.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~233s 2024-04-16T08:01:55.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~211s 2024-04-16T08:01:55.292 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~238s 2024-04-16T08:02:00.696 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~216s 2024-04-16T08:02:00.696 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~244s 2024-04-16T08:02:04.883 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:02:04.882+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T08:02:04.884 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:02:04.882+0000 7faf3e06b640 -1 calc_pg_upmaps abort due to max <= 0 2024-04-16T08:02:06.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~222s 2024-04-16T08:02:06.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~249s 2024-04-16T08:02:11.505 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~227s 2024-04-16T08:02:11.505 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~254s 2024-04-16T08:02:16.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~232s 2024-04-16T08:02:16.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~260s 2024-04-16T08:02:22.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~238s 2024-04-16T08:02:22.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~265s 2024-04-16T08:02:27.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~243s 2024-04-16T08:02:27.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~271s 2024-04-16T08:02:33.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~249s 2024-04-16T08:02:33.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~276s 2024-04-16T08:02:38.525 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~254s 2024-04-16T08:02:38.525 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~282s 2024-04-16T08:02:43.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~259s 2024-04-16T08:02:43.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~287s 2024-04-16T08:02:49.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~265s 2024-04-16T08:02:49.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~292s 2024-04-16T08:02:54.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~270s 2024-04-16T08:02:54.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~298s 2024-04-16T08:03:00.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~276s 2024-04-16T08:03:00.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~303s 2024-04-16T08:03:00.141 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-16T08:03:00.544 INFO:tasks.ceph.osd.1:Sent signal 15 2024-04-16T08:03:00.545 INFO:tasks.ceph.osd.2:Sent signal 15 2024-04-16T08:03:00.545 INFO:tasks.ceph.mon.a:Sent signal 15 2024-04-16T08:03:00.545 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-04-16T08:03:00.546 INFO:tasks.ceph.mon.a.smithi130.stderr:2024-04-16T08:03:00.546+0000 7f9442fff640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 31060) UID: 0 2024-04-16T08:03:00.546 INFO:tasks.ceph.mon.a.smithi130.stderr:2024-04-16T08:03:00.546+0000 7f9442fff640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-16T08:03:00.547 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:03:00.546+0000 7faf9e1f7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 30539) UID: 0 2024-04-16T08:03:00.547 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-16T08:03:00.546+0000 7faf9e1f7640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-16T10:51:03.031 INFO:tasks.workunit.client.0.smithi130.stderr:++ cleanup 2024-04-16T10:51:03.048 INFO:tasks.workunit.client.0.smithi130.stderr:++ pkill -P 33278 2024-04-16T10:51:03.051 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-16T10:51:03.052 INFO:tasks.workunit.client.0.smithi130.stderr:Terminated 2024-04-16T10:51:03.052 INFO:tasks.workunit.client.0.smithi130.stderr:++ true 2024-04-16T10:51:03.052 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-04-16T10:51:03.053 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-16T10:51:03.349 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi130 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7639cb6ce633dabc4d37769d62caacb54582984 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-04-16T10:51:03.538 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=38c72958cf9d4a1a8a2c67a9d4738b1d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi130 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7639cb6ce633dabc4d37769d62caacb54582984 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-04-16T10:51:03.541 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-16T10:51:03.550 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-16T10:51:03.550 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T10:51:03.668 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.665+0000 7ff6a71e0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-16T10:51:03.669 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.670+0000 7ff6a71e0640 1 Processor -- start 2024-04-16T10:51:03.669 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.670+0000 7ff6a71e0640 1 -- start start 2024-04-16T10:51:03.671 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.671+0000 7ff6a71e0640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6a0132060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:03.671 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.671+0000 7ff6a71e0640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:03.671 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.671+0000 7ff6a71e0640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:03.671 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.671+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6a0132060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:03.671 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.671+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff6a006eca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:03.671 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.671+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6a0132060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:03.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.872+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff6a006eca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:03.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.872+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6a0132060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:03.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:03.872+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6a0132060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:04.273+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6a0132060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:04.273+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff6a006eca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:04.272 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:04.273+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6a0132060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:05.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:05.074+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff6a006eca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:05.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:05.074+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6a0132060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:05.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:05.074+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6a0132060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:06.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6a0132060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:06.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6a0132060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:06.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff6a006eca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:06.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:06.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:06.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:06.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684007c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:06.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:06.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.671+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:06.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.872+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684007c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:06.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.872+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:06.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:06.872+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:07.272 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:07.273+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684007c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:07.272 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:07.273+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:07.272 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:07.273+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:08.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:08.074+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:08.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:08.074+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684007c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:08.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:08.074+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:09.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:09.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:09.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684007c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:09.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:09.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:09.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:09.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:09.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:09.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.672+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:09.871 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.873+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:09.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.873+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:09.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:09.873+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:10.274+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:10.274+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:10.273 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:10.274+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:11.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:11.075+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:11.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:11.075+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:11.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:11.075+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:12.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:12.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:12.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:12.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:12.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:12.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:12.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:12.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:12.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.673+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:12.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.874+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:12.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.874+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:12.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:12.874+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:13.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:13.275+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:13.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:13.275+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:13.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:13.275+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:14.076+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:14.076+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:14.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:14.076+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:15.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:15.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:15.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:15.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:15.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:15.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:15.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:15.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:15.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.672+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:15.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.873+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:15.872 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.873+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:15.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:15.873+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:16.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:16.274+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:16.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:16.274+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:16.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:16.274+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:17.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:17.075+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:17.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:17.075+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:17.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:17.075+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:18.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff6840056a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:18.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff6840056a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:18.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:18.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:18.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:18.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:18.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:18.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:18.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.673+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:18.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.874+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:18.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.874+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:18.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:18.874+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:19.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:19.275+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:19.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:19.275+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:19.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:19.275+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:20.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:20.077+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:20.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:20.077+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:20.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:20.077+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:21.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:21.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:21.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:21.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:21.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:21.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:21.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:21.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:21.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.673+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:21.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.874+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:21.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.874+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:21.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:21.874+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:22.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:22.275+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:22.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:22.275+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:22.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:22.275+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:23.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:23.076+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:23.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:23.076+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:23.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:23.076+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:24.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:24.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:24.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:24.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:24.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:24.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:24.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:24.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:24.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.673+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.874+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.874+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:24.873 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:24.874+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:25.274 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:25.275+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:25.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:25.275+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:25.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:25.275+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:26.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:26.076+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:26.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:26.076+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:26.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:26.076+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:27.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:27.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:27.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:27.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:27.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:27.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:27.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:27.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:27.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.674+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:27.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.875+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:27.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.875+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:27.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:27.875+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:28.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:28.276+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:28.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:28.276+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:28.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:28.276+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:29.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:29.077+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:29.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:29.077+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:29.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:29.077+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:30.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:30.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.674+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.875+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.875+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:30.875+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:31.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:31.276+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:31.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:31.276+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:31.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:31.276+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:32.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:32.077+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:32.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:32.077+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:32.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:32.077+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:33.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:33.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:33.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:33.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:33.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:33.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:33.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:33.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:33.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.674+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:33.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.875+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:33.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.875+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:33.874 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:33.875+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:34.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:34.276+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:34.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:34.276+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:34.275 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:34.276+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:35.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:35.077+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:35.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:35.077+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:35.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:35.077+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:36.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:36.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:36.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:36.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:36.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:36.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:36.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:36.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:36.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.675+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.876+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.876+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:36.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:36.876+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:37.276 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:37.277+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:37.276 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:37.277+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:37.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:37.277+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:38.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:38.078+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:38.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:38.078+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:38.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:38.078+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:39.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:39.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:39.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:39.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:39.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:39.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:39.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:39.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:39.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.675+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.876+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.876+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:39.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:39.876+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:40.276 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:40.277+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:40.276 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:40.277+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:40.276 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:40.277+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:41.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:41.078+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:41.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:41.078+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:41.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:41.078+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:42.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:42.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:42.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:42.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:42.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:42.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:42.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:42.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:42.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.675+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:42.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.876+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:42.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.876+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:42.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:42.876+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:43.276 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:43.277+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:43.276 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:43.277+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:43.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:43.277+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:44.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:44.078+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:44.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:44.078+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:44.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:44.078+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:45.675 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:45.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:45.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:45.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:45.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:45.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:45.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:45.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:45.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.676+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:45.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.877+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:45.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.877+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:45.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:45.877+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:46.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:46.278+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:46.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:46.278+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:46.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:46.278+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:47.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:47.079+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:47.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:47.079+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:47.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:47.079+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:48.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:48.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:48.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:48.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:48.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:48.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:48.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:48.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:48.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.676+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:48.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.877+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:48.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.877+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:48.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:48.877+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:49.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:49.278+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:49.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:49.278+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:49.277 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:49.278+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:50.078 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:50.079+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:50.079+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:50.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:50.079+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:51.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:51.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:51.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:51.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:51.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:51.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:51.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:51.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:51.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.677+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:51.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.878+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:51.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.878+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:51.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:51.878+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:52.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:52.279+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:52.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:52.279+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:52.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:52.279+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:53.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:53.080+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:53.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:53.080+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:53.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:53.080+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:54.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:54.676 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:51:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:54.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.677+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:54.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.878+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:54.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.878+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:54.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:54.878+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:55.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:55.279+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:55.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:55.279+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:55.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:55.279+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:56.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:56.080+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:56.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:56.080+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:56.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:56.080+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:51:57.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:57.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:51:57.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:51:57.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:51:57.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:51:57.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:51:57.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:57.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:57.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.677+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:51:57.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.878+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:57.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.878+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:57.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:57.878+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:51:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:58.279+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:58.279+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:58.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:58.279+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:51:59.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:59.080+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:51:59.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:59.080+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:51:59.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:51:59.080+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:00.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:00.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:00.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.677+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:00.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.878+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:00.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.878+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:00.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:00.878+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:01.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:01.279+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:01.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:01.279+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:01.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:01.279+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:02.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:02.080+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:02.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:02.080+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:02.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:02.080+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:03.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:03.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:03.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:03.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:03.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:03.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:03.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:03.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:03.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.678+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:03.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.879+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:03.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.879+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:03.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:03.879+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:04.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:04.280+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:04.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:04.280+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:04.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:04.280+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:05.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:05.081+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:05.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:05.081+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:05.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:05.081+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:06.677 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:06.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.677+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:06.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.878+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:06.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.878+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:06.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:06.878+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:07.278 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:07.279+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:07.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:07.279+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:07.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:07.279+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:08.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:08.080+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:08.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:08.080+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:08.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:08.080+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:09.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:09.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:09.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:09.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:09.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:09.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:09.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:09.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:09.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.678+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:09.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.879+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:09.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.879+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:09.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:09.879+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:10.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:10.280+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:10.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:10.280+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:10.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:10.280+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:11.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:11.081+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:11.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:11.081+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:11.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:11.081+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:12.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:12.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:12.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:12.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:12.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:12.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:12.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:12.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:12.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.678+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:12.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.879+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:12.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.879+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:12.878 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:12.879+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:13.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:13.280+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:13.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:13.280+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:13.279 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:13.280+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:14.081+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:14.081+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:14.080 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:14.081+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:15.678 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.678+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:15.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:15.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.678+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:15.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.678+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:15.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.678+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:15.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.678+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:15.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.679+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:15.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.679+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:15.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.679+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:15.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.880+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:15.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.880+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:15.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:15.880+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:16.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:16.281+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:16.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:16.281+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:16.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:16.281+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:17.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:17.082+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:17.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:17.082+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:17.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:17.082+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:18.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:18.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:18.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:18.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:18.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:18.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:18.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:18.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:18.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.679+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:18.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.880+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:18.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.880+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:18.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:18.880+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:19.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:19.281+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:19.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:19.281+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:19.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:19.281+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:20.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:20.082+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:20.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:20.082+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:20.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:20.082+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:21.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:21.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:21.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:21.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:21.679 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:21.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:21.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:21.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:21.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.679+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:21.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.880+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:21.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.880+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:21.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:21.880+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:22.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:22.281+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:22.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:22.281+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:22.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:22.281+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:23.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:23.082+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:23.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:23.082+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:23.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:23.082+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:24.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:24.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:24.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:24.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:24.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:24.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:24.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:24.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:24.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.679+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:24.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.880+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:24.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.880+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:24.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:24.880+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:25.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:25.281+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:25.280 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:25.281+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:25.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:25.281+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:26.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:26.082+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:26.081 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:26.082+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:26.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:26.082+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:27.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:27.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:27.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:27.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:27.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:27.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:27.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:27.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:27.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.680+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:27.880 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.881+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:27.880 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.881+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:27.880 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:27.881+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:28.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:28.282+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:28.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:28.282+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:28.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:28.282+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:29.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:29.083+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:29.083 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:29.083+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:29.083 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:29.083+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:30.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:30.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.680+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:30.880 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.881+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:30.880 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.881+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:30.880 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:30.881+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:31.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:31.282+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:31.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:31.282+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:31.281 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:31.282+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:32.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:32.083+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:32.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:32.083+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:32.082 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:32.083+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:33.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:33.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:33.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:33.680 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:33.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:33.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:33.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:33.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.680+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:33.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.681+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.882+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.882+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:33.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:33.882+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:34.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:34.283+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:34.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:34.283+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:34.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:34.283+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:35.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:35.084+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:35.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:35.084+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:35.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:35.084+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:36.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:36.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:36.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:36.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:36.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:36.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:36.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:36.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:36.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.681+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:36.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.882+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:36.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.882+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:36.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:36.882+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:37.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:37.283+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:37.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:37.283+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:37.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:37.283+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:38.083 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:38.084+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:38.083 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:38.084+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:38.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:38.084+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:39.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:39.681 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:39.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:39.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:39.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:39.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:39.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:39.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:39.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.681+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:39.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.882+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:39.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.882+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:39.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:39.882+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:40.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:40.283+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:40.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:40.283+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:40.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:40.283+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:41.083 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:41.084+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:41.083 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:41.084+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:41.083 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:41.084+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:42.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:42.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:42.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:42.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:42.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:42.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:42.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:42.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:42.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.682+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:42.881 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.883+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:42.882 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.883+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:42.882 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:42.883+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:43.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:43.284+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:43.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:43.284+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:43.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:43.284+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:44.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:44.085+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:44.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:44.085+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:44.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:44.085+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:45.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:45.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:45.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:45.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:45.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:45.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:45.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:45.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:45.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.682+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:45.882 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.883+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:45.882 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.883+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:45.882 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:45.883+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:46.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:46.284+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:46.282 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:46.284+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:46.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:46.284+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:47.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:47.085+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:47.084 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:47.085+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:47.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:47.085+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:48.682 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:48.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:48.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:48.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:48.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:48.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:48.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:48.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:48.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.683+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:48.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.884+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:48.884 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.884+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:48.884 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:48.884+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:49.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:49.285+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:49.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:49.285+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:49.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:49.285+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:50.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:50.086+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:50.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:50.086+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:50.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:50.086+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:51.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:51.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.683+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:51.882 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.884+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:51.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.884+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:51.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:51.884+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:52.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:52.285+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:52.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:52.285+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:52.285 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:52.285+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:53.086 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:53.086+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:53.086 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:53.086+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:53.086 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:53.086+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:54.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:52:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:54.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:54.686 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.683+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:54.882 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.884+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:54.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.884+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:54.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:54.884+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:55.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:55.284+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:55.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:55.284+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:55.283 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:55.284+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:56.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:56.085+0000 7ff6a4f7c640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:56.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:56.085+0000 7ff69ffff640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:56.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:56.085+0000 7ff6a4f7c640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:52:57.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:57.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:52:57.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:52:57.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:52:57.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009cd60 con 0x7ff6a006e890 2024-04-16T10:52:57.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:52:57.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:57.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:57.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.683+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:52:57.884 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.884+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:57.884 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.884+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:57.884 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:57.884+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:52:58.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:58.285+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:58.285 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:58.285+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:58.285 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:58.285+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:52:59.086 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:59.086+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:52:59.086 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:59.086+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:52:59.086 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:52:59.086+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:53:00.683 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69e7fc640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:53:00.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T10:53:00.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69e7fc640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T10:53:00.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69e7fc640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T10:53:00.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69e7fc640 1 -- --> v1:172.21.15.130:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a009b070 con 0x7ff6a006e890 2024-04-16T10:53:00.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69e7fc640 1 -- --> v2:172.21.15.130:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a0131800 con 0x7ff6a0131c50 2024-04-16T10:53:00.684 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:53:00.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:53:00.685 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.683+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T10:53:00.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.884+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:53:00.883 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.884+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:53:00.884 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:00.884+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T10:53:01.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:01.285+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:53:01.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:01.285+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:53:01.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:01.285+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T10:53:02.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:02.086+0000 7ff69ffff640 1 -- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 msgr2=0x7ff684005600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.130:3300/0 2024-04-16T10:53:02.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:02.086+0000 7ff6a4f7c640 1 -- >> v1:172.21.15.130:6789/0 conn(0x7ff6a006e890 legacy=0x7ff684009460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.130:6789/0 2024-04-16T10:53:02.085 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-16T10:53:02.086+0000 7ff69ffff640 1 --2- >> v2:172.21.15.130:3300/0 conn(0x7ff6a0131c50 0x7ff684005600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T10:53:03.589 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-16T10:53:03.590 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-16T10:53:03.599 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-16T10:53:03.600 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-16T10:53:03.600 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-16T10:53:03.600 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T10:53:03.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T10:53:03.602 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log' 2024-04-16T10:53:03.611 INFO:tasks.ceph.osd.0:Stopped 2024-04-16T10:53:03.612 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-16T10:53:03.612 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T10:53:03.677 INFO:tasks.ceph.osd.1:Stopped 2024-04-16T10:53:03.677 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-16T10:53:03.678 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T10:53:03.743 INFO:tasks.ceph.osd.2:Stopped 2024-04-16T10:53:03.743 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-16T10:53:03.743 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T10:53:03.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T10:53:03.743 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log' 2024-04-16T10:53:03.753 INFO:tasks.ceph.osd.3:Stopped 2024-04-16T10:53:03.753 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-16T10:53:03.753 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-16T10:53:03.753 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T10:53:03.755 INFO:tasks.ceph.mgr.x:Stopped 2024-04-16T10:53:03.755 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-16T10:53:03.755 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-16T10:53:03.756 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T10:53:03.756 INFO:tasks.ceph.mon.a:Stopped 2024-04-16T10:53:03.756 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-16T10:53:03.756 DEBUG:teuthology.orchestra.run.smithi130:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-04-16T10:53:03.791 INFO:teuthology.orchestra.run.smithi130.stdout:2024-04-16T07:58:17.859754+0000 mon.a (mon.0) 828 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-16T10:53:03.792 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-16T10:53:03.792 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi130.front.sepia.ceph.com 2024-04-16T10:53:03.792 DEBUG:teuthology.orchestra.run.smithi130:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-16T10:53:03.903 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi130.front.sepia.ceph.com 2024-04-16T10:53:03.903 DEBUG:teuthology.orchestra.run.smithi130:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-16T10:53:03.977 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-16T10:53:03.977 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-16T10:53:04.078 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-16T10:53:04.079 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-16T10:53:04.140 INFO:tasks.ceph:Archiving mon data... 2024-04-16T10:53:04.142 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/data/mon.a.tgz 2024-04-16T10:53:04.142 DEBUG:teuthology.orchestra.run.smithi130:> mktemp 2024-04-16T10:53:04.158 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/tmp.jnn3Q86wuQ 2024-04-16T10:53:04.158 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.jnn3Q86wuQ 2024-04-16T10:53:04.305 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0666 /tmp/tmp.jnn3Q86wuQ 2024-04-16T10:53:04.400 DEBUG:teuthology.orchestra.remote:smithi130:/tmp/tmp.jnn3Q86wuQ is 339KB 2024-04-16T10:53:04.496 DEBUG:teuthology.orchestra.run.smithi130:> rm -fr /tmp/tmp.jnn3Q86wuQ 2024-04-16T10:53:04.512 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-16T10:53:04.512 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T10:53:04.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T10:53:04.635 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T10:53:04.653 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T10:53:04.654 INFO:tasks.ceph:Archiving crash dumps... 2024-04-16T10:53:04.655 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/remote/smithi130/crash 2024-04-16T10:53:04.656 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T10:53:04.703 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/remote/smithi164/crash 2024-04-16T10:53:04.704 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T10:53:04.785 INFO:tasks.ceph:Compressing logs... 2024-04-16T10:53:04.785 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T10:53:04.787 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T10:53:04.813 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-16T10:53:04.813 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34521.log 2024-04-16T10:53:04.824 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30620.log 2024-04-16T10:53:04.824 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30700.log 2024-04-16T10:53:04.824 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30740.log 2024-04-16T10:53:04.824 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30620.log: /var/log/ceph/ceph-client.admin.30700.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.30620.log.gz 2024-04-16T10:53:04.824 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-16T10:53:04.825 INFO:teuthology.orchestra.run.smithi164.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.30700.log.gz 2024-04-16T10:53:04.825 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-16T10:53:04.825 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30740.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30740.log.gz 2024-04-16T10:53:04.825 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-16T10:53:04.825 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30660.log 2024-04-16T10:53:04.825 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.30660.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.30660.log.gz 2024-04-16T10:53:04.838 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32333.log 2024-04-16T10:53:04.838 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.34521.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28755.log 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32642.log 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32476.log 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32333.log.gz 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph.tmp-client.admin.28755.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34521.log.gz 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28755.log.gz 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31328.log 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 86.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32642.log.gz /var/log/ceph/ceph-client.admin.34689.log 2024-04-16T10:53:04.839 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32476.log: /var/log/ceph/ceph-client.admin.31328.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.32476.log.gz 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32793.log 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.31328.log.gz 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31063.log 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.34689.log: /var/log/ceph/ceph-client.admin.32793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32753.log 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.32793.log.gz 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.31063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33280.log 2024-04-16T10:53:04.840 INFO:teuthology.orchestra.run.smithi130.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.31063.log.gz 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32199.log 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32753.log: /var/log/ceph/ceph-client.admin.33280.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.32753.log.gz 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32879.log 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32474.log 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32199.log: /var/log/ceph/ceph-client.admin.32879.log: 86.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32477.log 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32199.log.gz 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32879.log.gz 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33047.log 2024-04-16T10:53:04.841 INFO:teuthology.orchestra.run.smithi130.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.32474.log.gz 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32963.log 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.33047.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.32477.log.gz 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34651.log 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.33047.log.gz 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32963.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32963.log.gz 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.34651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96336.log 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34651.log.gz 2024-04-16T10:53:04.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.96336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34574.log 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.34613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32475.log 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.34574.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34574.log.gz 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.33280.log.gz 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32644.log 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log/var/log/ceph/ceph-client.admin.32475.log: 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32644.log: gzip 86.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.34555.log 2024-04-16T10:53:04.843 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32475.log.gz 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.32644.log.gz 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip/var/log/ceph/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32899.log 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.34555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34670.log 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32899.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32899.log.gz 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96843.log 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.34670.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.34555.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34670.log.gz 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32643.log 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.96843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32792.log 2024-04-16T10:53:04.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32643.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32643.log.gz 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96446.log 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.96843.log.gz 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32792.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32792.log.gz 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32249.log 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.96446.log: -- /var/log/ceph/ceph-client.admin.96408.log 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32375.log 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32249.log.gz 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.96408.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96408.log.gz 2024-04-16T10:53:04.845 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96627.log 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.96446.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32375.log: /var/log/ceph/ceph-client.admin.96627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31299.log 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-osd.0.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.32375.log.gz 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33005.log 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.96336.log.gz 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.31299.log: 85.0%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31299.log.gz 2024-04-16T10:53:04.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.33005.log: gzip 85.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.33749.log 2024-04-16T10:53:04.847 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33005.log.gz 2024-04-16T10:53:04.847 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32143.log 2024-04-16T10:53:04.847 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.33749.log: 88.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-16T10:53:04.847 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32645.log 2024-04-16T10:53:04.847 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32143.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32143.log.gz 2024-04-16T10:53:04.847 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31142.log 2024-04-16T10:53:04.847 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32645.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.32645.log.gz 2024-04-16T10:53:04.848 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32291.log 2024-04-16T10:53:04.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.31142.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.31142.log.gz 2024-04-16T10:53:04.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.admin.32291.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.32291.log.gz 2024-04-16T10:53:04.849 INFO:teuthology.orchestra.run.smithi130.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.33749.log.gz 2024-04-16T10:53:04.860 INFO:teuthology.orchestra.run.smithi130.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.admin.96627.log.gz 2024-04-16T10:53:04.994 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-16T10:53:05.263 INFO:teuthology.orchestra.run.smithi130.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.34689.log.gz 2024-04-16T10:53:06.096 INFO:teuthology.orchestra.run.smithi130.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-16T10:53:06.599 INFO:teuthology.orchestra.run.smithi164.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-16T10:53:06.715 INFO:teuthology.orchestra.run.smithi164.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-16T10:53:06.717 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-16T10:53:06.717 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m1.913s 2024-04-16T10:53:06.717 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m3.680s 2024-04-16T10:53:06.717 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.199s 2024-04-16T10:53:07.045 INFO:teuthology.orchestra.run.smithi130.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-16T10:53:07.075 INFO:teuthology.orchestra.run.smithi130.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-16T10:53:07.077 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-16T10:53:07.077 INFO:teuthology.orchestra.run.smithi130.stderr:real 0m2.277s 2024-04-16T10:53:07.077 INFO:teuthology.orchestra.run.smithi130.stderr:user 0m6.045s 2024-04-16T10:53:07.077 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m0.285s 2024-04-16T10:53:07.077 INFO:tasks.ceph:Archiving logs... 2024-04-16T10:53:07.078 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/remote/smithi130/log 2024-04-16T10:53:07.079 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T10:53:08.015 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/remote/smithi164/log 2024-04-16T10:53:08.016 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T10:53:08.616 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-16T10:53:08.625 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-16T10:53:08.634 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-16T10:53:08.638 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-16T10:53:08.639 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T10:53:08.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T10:53:08.725 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-16T10:53:08.733 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-16T10:53:08.733 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T10:53:08.736 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T10:53:08.750 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-16T10:53:08.753 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-16T10:53:08.755 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T10:53:08.756 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-16T10:53:08.756 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv01.front.sepia.ceph.com 3 7 377 1 +397us[ +500us] +/- 62ms 2024-04-16T10:53:08.756 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 125 -1299us[-1198us] +/- 73ms 2024-04-16T10:53:08.756 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +730us[ +832us] +/- 69ms 2024-04-16T10:53:08.756 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T10:53:08.757 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T10:53:08.757 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-16T10:53:08.757 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +438us[ +429us] +/- 61ms 2024-04-16T10:53:08.758 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1260us[-1268us] +/- 74ms 2024-04-16T10:53:08.758 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +810us[ +801us] +/- 69ms 2024-04-16T10:53:08.758 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T10:53:08.759 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-16T10:53:08.767 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-16T10:53:08.768 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-16T10:53:08.777 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T10:53:08.870 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T10:53:08.941 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T10:53:08.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T10:53:08.973 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T10:53:09.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T10:53:09.006 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-16T10:53:09.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-16T10:53:09.026 INFO:teuthology.task.internal:Duration was 11776.789851 seconds 2024-04-16T10:53:09.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-16T10:53:09.036 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-16T10:53:09.037 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T10:53:09.039 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T10:53:09.084 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T10:53:09.476 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-16T10:53:09.476 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-04-16T10:53:09.476 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T10:53:09.497 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-04-16T10:53:09.497 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T10:53:09.524 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-16T10:53:09.524 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T10:53:09.540 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T10:53:09.594 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-16T10:53:09.594 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T10:53:09.609 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T10:53:09.735 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-16T10:53:09.744 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-16T10:53:09.744 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T10:53:09.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T10:53:09.801 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-16T10:53:09.813 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T10:53:09.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T10:53:09.840 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-04-16T10:53:09.868 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-04-16T10:53:09.889 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T10:53:09.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T10:53:09.919 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T10:53:09.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T10:53:09.945 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-16T10:53:09.954 INFO:teuthology.task.internal:Transferring archived files... 2024-04-16T10:53:09.955 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/remote/smithi130 2024-04-16T10:53:09.955 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T10:53:09.995 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979/remote/smithi164 2024-04-16T10:53:09.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T10:53:10.031 INFO:teuthology.task.internal:Removing archive directory... 2024-04-16T10:53:10.031 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T10:53:10.034 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T10:53:10.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-16T10:53:10.092 INFO:teuthology.task.internal:Not uploading archives. 2024-04-16T10:53:10.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-16T10:53:10.106 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-16T10:53:10.106 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T10:53:10.108 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T10:53:10.121 INFO:teuthology.orchestra.run.smithi130.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 16 10:53 /home/ubuntu/cephtest 2024-04-16T10:53:10.121 INFO:teuthology.orchestra.run.smithi130.stdout: 403413 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 16 07:50 /home/ubuntu/cephtest/mnt.0 2024-04-16T10:53:10.121 INFO:teuthology.orchestra.run.smithi130.stdout: 403414 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 16 07:51 /home/ubuntu/cephtest/mnt.0/client.0 2024-04-16T10:53:10.122 INFO:teuthology.orchestra.run.smithi130.stdout: 417793 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 07:51 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-16T10:53:10.123 INFO:teuthology.orchestra.run.smithi130.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-16T10:53:10.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T10:53:10.123 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph_a7639cb6ce633dabc4d37769d62caacb54582984/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi130 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-16T10:53:10.136 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-16T10:53:10.145 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-16T10:53:10.161 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi130 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-16T10:53:10.161 INFO:teuthology.nuke:Checking targets against current locks 2024-04-16T10:53:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 10:53 /home/ubuntu/cephtest 2024-04-16T10:53:10.190 DEBUG:teuthology.nuke:shortname: smithi130 2024-04-16T10:53:10.190 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T10:53:10.213 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 07:27:14.632469', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ3frryKjjuLRnl70qaclrLNafzBX76iNXxFw3R1ss5XsTmwjUsliogWn4rfLFlamMR2t9AaHgOzLQU2HkbJd8s='} 2024-04-16T10:53:10.216 DEBUG:teuthology.nuke:shortname: smithi164 2024-04-16T10:53:10.217 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T10:53:10.238 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-16_07:26:08-crimson-rados-wip-matanb-crimson-replicated-april-15-distro-crimson-smithi/7657979', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 07:27:14.634229', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCAVTSCYMgv1PQYlBnurqYd7NhPHCnsHcIcI6lKQiTWoqpmW7iVCioIXHo5yYKnvin+Ne7wIGNkWxplxHDc58s='} 2024-04-16T10:53:10.270 INFO:teuthology.orchestra.console.smithi130:Power off 2024-04-16T10:53:10.270 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T10:53:10.292 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2024-04-16T10:53:10.293 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T10:53:10.307 INFO:teuthology.orchestra.console.smithi164:Power off 2024-04-16T10:53:10.307 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T10:53:10.315 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-16T10:53:10.327 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-04-16T10:53:10.327 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T10:53:10.348 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-16T10:53:14.317 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T10:53:14.349 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T10:53:14.430 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-16T10:53:14.461 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-16T10:53:18.436 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T10:53:18.462 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T10:53:18.550 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-16T10:53:18.574 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-04-16T10:53:18.675 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-04-16T10:53:22.555 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T10:53:22.667 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2024-04-16T10:53:22.768 INFO:teuthology.orchestra.console.smithi130:Power off completed 2024-04-16T10:53:22.873 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 11776.789851427078 failure_reason: 'Command failed (workunit test rados/test.sh) on smithi130 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7639cb6ce633dabc4d37769d62caacb54582984 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson''' flavor: crimson owner: scheduled_matan@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=38c72958cf9d4a1a8a2c67a9d4738b1d status: fail success: false 2024-04-16T10:53:22.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T10:53:22.951 INFO:teuthology.run:FAIL