2023-10-12T02:00:12.869 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-12T02:00:12.870 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yingxin@teuthology --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} --name yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi --verbose --archive /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367 -- /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/orig.config.yaml 2023-10-12T02:00:12.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T02:00:12.984 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367 branch: wip-yingxin-crimson-osd-crosscore-pg-submission description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: yingxin.cheng@intel.com first_in_suite: false job_id: '7422367' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yingxin-crimson-osd-crosscore-pg-submission ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson seastar smp: 3 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: crimson sha1: 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 workunit: branch: wip-yingxin-crimson-osd-crosscore-pg-submission env: CRIMSON_COMPAT: '1' sha1: 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 owner: scheduled_yingxin@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b sha1: 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-yingxin-crimson-osd-crosscore-pg-submission suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 targets: smithi125.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN73DjrdE1g/zWHtmfxoMBCa2KEZsi38QXldujS7u0Zww7nZEB2UwjO8NtmyYYBxfjV9JEweM2TVFCIb8a4Mc+8= smithi158.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCBwCQt/Ud2a8XaZeiwNWB9DezxlH0sYlAKXUvpN15HLpscSYamtBdB1EmEAaFCHN7visu6WEigz8M/5Vw1NYI= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-12_01:35:43 tube: smithi user: yingxin verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-12T02:00:12.984 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa; will attempt to use it 2023-10-12T02:00:12.985 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks 2023-10-12T02:00:12.985 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-12T02:00:12.987 INFO:teuthology.task.internal:Checking packages... 2023-10-12T02:00:13.013 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5' 2023-10-12T02:00:13.014 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-12T02:00:13.014 INFO:teuthology.packaging:ref: None 2023-10-12T02:00:13.014 INFO:teuthology.packaging:tag: None 2023-10-12T02:00:13.015 INFO:teuthology.packaging:branch: wip-yingxin-crimson-osd-crosscore-pg-submission 2023-10-12T02:00:13.015 INFO:teuthology.packaging:sha1: 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 2023-10-12T02:00:13.015 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-yingxin-crimson-osd-crosscore-pg-submission 2023-10-12T02:00:13.157 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6684.g9587ffdf 2023-10-12T02:00:13.157 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-12T02:00:13.164 INFO:teuthology.task.internal:no buildpackages task found 2023-10-12T02:00:13.164 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-12T02:00:13.179 INFO:teuthology.task.internal:Saving configuration 2023-10-12T02:00:13.191 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-12T02:00:13.198 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-12T02:00:13.226 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-12 01:54:32.373950', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN73DjrdE1g/zWHtmfxoMBCa2KEZsi38QXldujS7u0Zww7nZEB2UwjO8NtmyYYBxfjV9JEweM2TVFCIb8a4Mc+8='} 2023-10-12T02:00:13.246 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-12 01:54:32.375532', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCBwCQt/Ud2a8XaZeiwNWB9DezxlH0sYlAKXUvpN15HLpscSYamtBdB1EmEAaFCHN7visu6WEigz8M/5Vw1NYI='} 2023-10-12T02:00:13.247 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-12T02:00:13.253 INFO:teuthology.task.internal:roles: ubuntu@smithi125.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2023-10-12T02:00:13.254 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-10-12T02:00:13.254 INFO:teuthology.run_tasks:Running task console_log... 2023-10-12T02:00:13.326 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f232e19e940>, signals=[15]) 2023-10-12T02:00:13.328 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-12T02:00:13.334 INFO:teuthology.task.internal:Opening connections... 2023-10-12T02:00:13.334 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi125.front.sepia.ceph.com 2023-10-12T02:00:13.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:00:13.422 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi158.front.sepia.ceph.com 2023-10-12T02:00:13.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:00:13.505 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-12T02:00:13.512 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2023-10-12T02:00:13.530 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2023-10-12T02:00:13.531 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-10-12T02:00:13.588 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-10-12T02:00:13.588 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2023-10-12T02:00:13.589 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-10-12T02:00:13.589 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-10-12T02:00:13.589 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2023-10-12T02:00:13.589 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2023-10-12T02:00:13.589 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-12T02:00:13.590 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-10-12T02:00:13.590 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2023-10-12T02:00:13.590 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-12T02:00:13.590 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-10-12T02:00:13.591 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-12T02:00:13.591 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-12T02:00:13.591 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-12T02:00:13.592 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-10-12T02:00:13.613 DEBUG:teuthology.orchestra.run.smithi158:> uname -m 2023-10-12T02:00:13.631 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64 2023-10-12T02:00:13.632 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2023-10-12T02:00:13.688 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream" 2023-10-12T02:00:13.688 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9" 2023-10-12T02:00:13.688 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos" 2023-10-12T02:00:13.688 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora" 2023-10-12T02:00:13.689 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9" 2023-10-12T02:00:13.689 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9" 2023-10-12T02:00:13.689 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-12T02:00:13.689 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2023-10-12T02:00:13.689 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon" 2023-10-12T02:00:13.690 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-12T02:00:13.690 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/" 2023-10-12T02:00:13.690 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-12T02:00:13.690 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-12T02:00:13.691 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-12T02:00:13.691 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2023-10-12T02:00:13.713 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-12T02:00:13.722 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-12T02:00:13.733 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-12T02:00:13.733 DEBUG:teuthology.orchestra.run.smithi125:> test '!' -e /home/ubuntu/cephtest 2023-10-12T02:00:13.737 DEBUG:teuthology.orchestra.run.smithi158:> test '!' -e /home/ubuntu/cephtest 2023-10-12T02:00:13.752 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-12T02:00:13.758 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-12T02:00:13.758 DEBUG:teuthology.orchestra.run.smithi125:> test -z $(ls -A /var/lib/ceph) 2023-10-12T02:00:13.793 DEBUG:teuthology.orchestra.run.smithi158:> test -z $(ls -A /var/lib/ceph) 2023-10-12T02:00:13.900 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-12T02:00:13.954 INFO:teuthology.run_tasks:Running task kernel... 2023-10-12T02:00:13.968 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-12T02:00:13.969 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-12T02:00:13.970 DEBUG:teuthology.orchestra.run.smithi125:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-12T02:00:13.970 DEBUG:teuthology.orchestra.run.smithi158:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-12T02:00:13.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T02:00:13.987 DEBUG:teuthology.orchestra.run.smithi158:> uname -r 2023-10-12T02:00:13.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T02:00:13.988 DEBUG:teuthology.orchestra.run.smithi125:> uname -r 2023-10-12T02:00:14.043 INFO:teuthology.orchestra.run.smithi158.stdout:5.14.0-361.el9.x86_64 2023-10-12T02:00:14.044 INFO:teuthology.task.kernel:Running kernel on smithi158: 5.14.0-361.el9.x86_64 2023-10-12T02:00:14.044 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum install -y kernel 2023-10-12T02:00:14.046 INFO:teuthology.orchestra.run.smithi125.stdout:5.14.0-361.el9.x86_64 2023-10-12T02:00:14.046 INFO:teuthology.task.kernel:Running kernel on smithi125: 5.14.0-361.el9.x86_64 2023-10-12T02:00:14.046 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum install -y kernel 2023-10-12T02:00:16.162 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-10-12T02:00:16.251 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-10-12T02:00:17.470 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2023-10-12T02:00:17.894 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2023-10-12T02:00:18.936 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 7.9 MB 00:01 2023-10-12T02:00:19.926 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 7.9 MB 00:01 2023-10-12T02:00:20.853 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - AppStream 20 MB/s | 18 MB 00:00 2023-10-12T02:00:22.177 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - AppStream 14 MB/s | 18 MB 00:01 2023-10-12T02:00:25.337 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 5.7 MB 00:01 2023-10-12T02:00:26.798 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 14 kB 00:00 2023-10-12T02:00:27.253 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 5.7 MB 00:01 2023-10-12T02:00:28.720 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 14 kB 00:00 2023-10-12T02:00:29.574 INFO:teuthology.orchestra.run.smithi158.stdout:Extra Packages for Enterprise Linux 25 MB/s | 19 MB 00:00 2023-10-12T02:00:30.592 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 19 MB 00:03 2023-10-12T02:00:34.063 INFO:teuthology.orchestra.run.smithi158.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-10-12T02:00:35.088 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-10-12T02:00:36.095 INFO:teuthology.orchestra.run.smithi158.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-12T02:00:36.095 INFO:teuthology.orchestra.run.smithi158.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-12T02:00:36.194 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:00:36.203 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:00:36.203 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:00:36.203 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:00:36.204 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:00:36.204 INFO:teuthology.orchestra.run.smithi158.stdout: kernel x86_64 5.14.0-375.el9 CentOS-BaseOS 5.3 M 2023-10-12T02:00:36.204 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:00:36.204 INFO:teuthology.orchestra.run.smithi158.stdout: kernel-core x86_64 5.14.0-375.el9 CentOS-BaseOS 20 M 2023-10-12T02:00:36.205 INFO:teuthology.orchestra.run.smithi158.stdout: kernel-modules x86_64 5.14.0-375.el9 CentOS-BaseOS 37 M 2023-10-12T02:00:36.205 INFO:teuthology.orchestra.run.smithi158.stdout: kernel-modules-core x86_64 5.14.0-375.el9 CentOS-BaseOS 32 M 2023-10-12T02:00:36.205 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:00:36.205 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:00:36.205 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:00:36.206 INFO:teuthology.orchestra.run.smithi158.stdout:Install 4 Packages 2023-10-12T02:00:36.206 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:00:36.207 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 95 M 2023-10-12T02:00:36.207 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 120 M 2023-10-12T02:00:36.207 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:00:36.515 INFO:teuthology.orchestra.run.smithi158.stdout:(1/4): kernel-5.14.0-375.el9.x86_64.rpm 17 MB/s | 5.3 MB 00:00 2023-10-12T02:00:37.040 INFO:teuthology.orchestra.run.smithi158.stdout:(2/4): kernel-core-5.14.0-375.el9.x86_64.rpm 24 MB/s | 20 MB 00:00 2023-10-12T02:00:37.128 INFO:teuthology.orchestra.run.smithi125.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-12T02:00:37.128 INFO:teuthology.orchestra.run.smithi125.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-12T02:00:37.222 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:00:37.233 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:00:37.233 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:00:37.234 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:00:37.234 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:00:37.234 INFO:teuthology.orchestra.run.smithi125.stdout: kernel x86_64 5.14.0-375.el9 CentOS-BaseOS 5.3 M 2023-10-12T02:00:37.234 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:00:37.235 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-core x86_64 5.14.0-375.el9 CentOS-BaseOS 20 M 2023-10-12T02:00:37.235 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules x86_64 5.14.0-375.el9 CentOS-BaseOS 37 M 2023-10-12T02:00:37.235 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-core x86_64 5.14.0-375.el9 CentOS-BaseOS 32 M 2023-10-12T02:00:37.235 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:00:37.235 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:00:37.236 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:00:37.236 INFO:teuthology.orchestra.run.smithi125.stdout:Install 4 Packages 2023-10-12T02:00:37.236 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:00:37.237 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 95 M 2023-10-12T02:00:37.237 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 120 M 2023-10-12T02:00:37.237 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:00:37.573 INFO:teuthology.orchestra.run.smithi125.stdout:(1/4): kernel-5.14.0-375.el9.x86_64.rpm 16 MB/s | 5.3 MB 00:00 2023-10-12T02:00:37.665 INFO:teuthology.orchestra.run.smithi158.stdout:(3/4): kernel-modules-5.14.0-375.el9.x86_64.rpm 26 MB/s | 37 MB 00:01 2023-10-12T02:00:38.165 INFO:teuthology.orchestra.run.smithi125.stdout:(2/4): kernel-core-5.14.0-375.el9.x86_64.rpm 21 MB/s | 20 MB 00:00 2023-10-12T02:00:38.182 INFO:teuthology.orchestra.run.smithi158.stdout:(4/4): kernel-modules-core-5.14.0-375.el9.x86_6 19 MB/s | 32 MB 00:01 2023-10-12T02:00:38.186 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:00:38.186 INFO:teuthology.orchestra.run.smithi158.stdout:Total 48 MB/s | 95 MB 00:01 2023-10-12T02:00:38.186 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:00:38.546 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:00:38.546 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:00:38.782 INFO:teuthology.orchestra.run.smithi125.stdout:(3/4): kernel-modules-5.14.0-375.el9.x86_64.rpm 24 MB/s | 37 MB 00:01 2023-10-12T02:00:39.065 INFO:teuthology.orchestra.run.smithi125.stdout:(4/4): kernel-modules-core-5.14.0-375.el9.x86_6 22 MB/s | 32 MB 00:01 2023-10-12T02:00:39.068 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi125.stdout:Total 52 MB/s | 95 MB 00:01 2023-10-12T02:00:39.069 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:00:39.409 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:00:39.409 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:00:39.519 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:00:39.519 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:00:40.382 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:00:40.383 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:00:41.903 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:00:42.595 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : kernel-modules-core-5.14.0-375.el9.x86_64 1/4 2023-10-12T02:00:42.727 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : kernel-core-5.14.0-375.el9.x86_64 2/4 2023-10-12T02:00:42.752 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:00:43.421 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-modules-core-5.14.0-375.el9.x86_64 1/4 2023-10-12T02:00:43.540 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-core-5.14.0-375.el9.x86_64 2/4 2023-10-12T02:00:45.478 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: kernel-core-5.14.0-375.el9.x86_64 2/4 2023-10-12T02:00:45.664 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : kernel-modules-5.14.0-375.el9.x86_64 3/4 2023-10-12T02:00:46.283 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-core-5.14.0-375.el9.x86_64 2/4 2023-10-12T02:00:46.458 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-modules-5.14.0-375.el9.x86_64 3/4 2023-10-12T02:00:50.360 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: kernel-modules-5.14.0-375.el9.x86_64 3/4 2023-10-12T02:00:50.465 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : kernel-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:00:51.093 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-5.14.0-375.el9.x86_64 3/4 2023-10-12T02:00:51.476 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:00:55.088 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: kernel-modules-core-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:00:56.094 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-core-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:24.218 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: kernel-core-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:24.261 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: kernel-modules-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:24.620 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: kernel-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:24.621 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : kernel-5.14.0-375.el9.x86_64 1/4 2023-10-12T02:01:24.621 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : kernel-core-5.14.0-375.el9.x86_64 2/4 2023-10-12T02:01:24.621 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : kernel-modules-5.14.0-375.el9.x86_64 3/4 2023-10-12T02:01:24.880 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-core-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:24.922 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:25.046 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : kernel-modules-core-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:25.047 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:01:25.047 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:01:25.047 INFO:teuthology.orchestra.run.smithi158.stdout: kernel-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.047 INFO:teuthology.orchestra.run.smithi158.stdout: kernel-core-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.048 INFO:teuthology.orchestra.run.smithi158.stdout: kernel-modules-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.048 INFO:teuthology.orchestra.run.smithi158.stdout: kernel-modules-core-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.048 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:01:25.048 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:01:25.301 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:25.301 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-5.14.0-375.el9.x86_64 1/4 2023-10-12T02:01:25.302 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-core-5.14.0-375.el9.x86_64 2/4 2023-10-12T02:01:25.302 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-modules-5.14.0-375.el9.x86_64 3/4 2023-10-12T02:01:25.578 DEBUG:teuthology.orchestra.run.smithi158:> rpm -q kernel | sort -rV | head -n 1 2023-10-12T02:01:25.708 INFO:teuthology.orchestra.run.smithi158.stdout:kernel-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.708 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-375.el9.x86_64 2023-10-12T02:01:25.709 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-375.el9.x86_64 2023-10-12T02:01:25.709 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-12T02:01:25.709 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-12T02:01:25.709 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2023-10-12T02:01:25.710 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi158.front.sepia.ceph.com, path=None, version=distro) 2023-10-12T02:01:25.710 DEBUG:teuthology.orchestra.run.smithi158:> rpm -q kernel | sort -rV | head -n 1 2023-10-12T02:01:25.740 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-modules-core-5.14.0-375.el9.x86_64 4/4 2023-10-12T02:01:25.740 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:01:25.740 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:01:25.740 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.741 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-core-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.741 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.741 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-core-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.741 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:01:25.741 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:01:25.815 INFO:teuthology.orchestra.run.smithi158.stdout:kernel-5.14.0-375.el9.x86_64 2023-10-12T02:01:25.815 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-375.el9.x86_64 2023-10-12T02:01:25.815 DEBUG:teuthology.orchestra.run.smithi158:> sudo rpm -qi grub2-tools 2023-10-12T02:01:25.858 INFO:teuthology.orchestra.run.smithi158.stdout:Name : grub2-tools 2023-10-12T02:01:25.858 INFO:teuthology.orchestra.run.smithi158.stdout:Epoch : 1 2023-10-12T02:01:25.858 INFO:teuthology.orchestra.run.smithi158.stdout:Version : 2.06 2023-10-12T02:01:25.859 INFO:teuthology.orchestra.run.smithi158.stdout:Release : 68.el9 2023-10-12T02:01:25.859 INFO:teuthology.orchestra.run.smithi158.stdout:Architecture: x86_64 2023-10-12T02:01:25.859 INFO:teuthology.orchestra.run.smithi158.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-12T02:01:25.859 INFO:teuthology.orchestra.run.smithi158.stdout:Group : Unspecified 2023-10-12T02:01:25.860 INFO:teuthology.orchestra.run.smithi158.stdout:Size : 8264254 2023-10-12T02:01:25.860 INFO:teuthology.orchestra.run.smithi158.stdout:License : GPLv3+ 2023-10-12T02:01:25.860 INFO:teuthology.orchestra.run.smithi158.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-12T02:01:25.860 INFO:teuthology.orchestra.run.smithi158.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-12T02:01:25.860 INFO:teuthology.orchestra.run.smithi158.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-12T02:01:25.861 INFO:teuthology.orchestra.run.smithi158.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-12T02:01:25.861 INFO:teuthology.orchestra.run.smithi158.stdout:Packager : builder@centos.org 2023-10-12T02:01:25.861 INFO:teuthology.orchestra.run.smithi158.stdout:Vendor : CentOS 2023-10-12T02:01:25.861 INFO:teuthology.orchestra.run.smithi158.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-12T02:01:25.861 INFO:teuthology.orchestra.run.smithi158.stdout:Summary : Support tools for GRUB. 2023-10-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi158.stdout:Description : 2023-10-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi158.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi158.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-12T02:01:25.862 INFO:teuthology.orchestra.run.smithi158.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi158.stdout:hardware devices. 2023-10-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:01:25.863 INFO:teuthology.orchestra.run.smithi158.stdout:This subpackage provides tools for support of all platforms. 2023-10-12T02:01:25.864 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-12T02:01:25.864 INFO:teuthology.task.kernel:Updating grub on smithi158 to boot 5.14.0-375.el9.x86_64 2023-10-12T02:01:25.864 DEBUG:teuthology.orchestra.run.smithi158:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-12T02:01:26.238 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q kernel | sort -rV | head -n 1 2023-10-12T02:01:26.371 INFO:teuthology.orchestra.run.smithi125.stdout:kernel-5.14.0-375.el9.x86_64 2023-10-12T02:01:26.371 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-375.el9.x86_64 2023-10-12T02:01:26.371 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-375.el9.x86_64 2023-10-12T02:01:26.372 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-12T02:01:26.372 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-12T02:01:26.372 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-12T02:01:26.372 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi125.front.sepia.ceph.com, path=None, version=distro) 2023-10-12T02:01:26.373 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q kernel | sort -rV | head -n 1 2023-10-12T02:01:26.477 INFO:teuthology.orchestra.run.smithi125.stdout:kernel-5.14.0-375.el9.x86_64 2023-10-12T02:01:26.477 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-375.el9.x86_64 2023-10-12T02:01:26.478 DEBUG:teuthology.orchestra.run.smithi125:> sudo rpm -qi grub2-tools 2023-10-12T02:01:26.482 INFO:teuthology.orchestra.run.smithi158.stderr:Generating grub configuration file ... 2023-10-12T02:01:26.515 INFO:teuthology.orchestra.run.smithi125.stdout:Name : grub2-tools 2023-10-12T02:01:26.516 INFO:teuthology.orchestra.run.smithi125.stdout:Epoch : 1 2023-10-12T02:01:26.516 INFO:teuthology.orchestra.run.smithi125.stdout:Version : 2.06 2023-10-12T02:01:26.516 INFO:teuthology.orchestra.run.smithi125.stdout:Release : 68.el9 2023-10-12T02:01:26.517 INFO:teuthology.orchestra.run.smithi125.stdout:Architecture: x86_64 2023-10-12T02:01:26.517 INFO:teuthology.orchestra.run.smithi125.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-12T02:01:26.517 INFO:teuthology.orchestra.run.smithi125.stdout:Group : Unspecified 2023-10-12T02:01:26.517 INFO:teuthology.orchestra.run.smithi125.stdout:Size : 8264254 2023-10-12T02:01:26.517 INFO:teuthology.orchestra.run.smithi125.stdout:License : GPLv3+ 2023-10-12T02:01:26.518 INFO:teuthology.orchestra.run.smithi125.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-12T02:01:26.518 INFO:teuthology.orchestra.run.smithi125.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-12T02:01:26.518 INFO:teuthology.orchestra.run.smithi125.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-12T02:01:26.518 INFO:teuthology.orchestra.run.smithi125.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-12T02:01:26.518 INFO:teuthology.orchestra.run.smithi125.stdout:Packager : builder@centos.org 2023-10-12T02:01:26.519 INFO:teuthology.orchestra.run.smithi125.stdout:Vendor : CentOS 2023-10-12T02:01:26.519 INFO:teuthology.orchestra.run.smithi125.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-12T02:01:26.519 INFO:teuthology.orchestra.run.smithi125.stdout:Summary : Support tools for GRUB. 2023-10-12T02:01:26.519 INFO:teuthology.orchestra.run.smithi125.stdout:Description : 2023-10-12T02:01:26.519 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:01:26.520 INFO:teuthology.orchestra.run.smithi125.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-12T02:01:26.520 INFO:teuthology.orchestra.run.smithi125.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-12T02:01:26.520 INFO:teuthology.orchestra.run.smithi125.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-12T02:01:26.520 INFO:teuthology.orchestra.run.smithi125.stdout:hardware devices. 2023-10-12T02:01:26.520 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:01:26.521 INFO:teuthology.orchestra.run.smithi125.stdout:This subpackage provides tools for support of all platforms. 2023-10-12T02:01:26.521 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-12T02:01:26.522 INFO:teuthology.task.kernel:Updating grub on smithi125 to boot 5.14.0-375.el9.x86_64 2023-10-12T02:01:26.522 DEBUG:teuthology.orchestra.run.smithi125:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-12T02:01:27.235 INFO:teuthology.orchestra.run.smithi125.stderr:Generating grub configuration file ... 2023-10-12T02:01:27.873 INFO:teuthology.orchestra.run.smithi158.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-12T02:01:27.935 INFO:teuthology.orchestra.run.smithi158.stderr:done 2023-10-12T02:01:27.937 DEBUG:teuthology.orchestra.run.smithi158:> mktemp 2023-10-12T02:01:27.953 INFO:teuthology.orchestra.run.smithi158.stdout:/tmp/tmp.jYH9PHlBKt 2023-10-12T02:01:27.953 DEBUG:teuthology.orchestra.run.smithi158:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jYH9PHlBKt 2023-10-12T02:01:28.021 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 0666 /tmp/tmp.jYH9PHlBKt 2023-10-12T02:01:28.162 DEBUG:teuthology.orchestra.remote:smithi158:/tmp/tmp.jYH9PHlBKt is 6KB 2023-10-12T02:01:28.210 DEBUG:teuthology.orchestra.run.smithi158:> rm -fr /tmp/tmp.jYH9PHlBKt 2023-10-12T02:01:28.224 DEBUG:teuthology.orchestra.run.smithi158:> sudo /bin/ls /boot/loader/entries || true 2023-10-12T02:01:28.289 INFO:teuthology.orchestra.run.smithi158.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-12T02:01:28.290 INFO:teuthology.orchestra.run.smithi158.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-12T02:01:28.290 INFO:teuthology.orchestra.run.smithi158.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-12T02:01:28.290 INFO:teuthology.orchestra.run.smithi158.stdout:15949e3c88704300882482eddfd7baae-5.14.0-375.el9.x86_64.conf 2023-10-12T02:01:28.291 DEBUG:teuthology.orchestra.run.smithi158:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-375.el9.x86_64 2023-10-12T02:01:28.447 DEBUG:teuthology.orchestra.run.smithi158:> sudo shutdown -r now 2023-10-12T02:01:28.573 INFO:teuthology.orchestra.run.smithi125.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-12T02:01:28.606 INFO:teuthology.orchestra.run.smithi125.stderr:done 2023-10-12T02:01:28.608 DEBUG:teuthology.orchestra.run.smithi125:> mktemp 2023-10-12T02:01:28.623 INFO:teuthology.orchestra.run.smithi125.stdout:/tmp/tmp.6ovGwPOVNd 2023-10-12T02:01:28.624 DEBUG:teuthology.orchestra.run.smithi125:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6ovGwPOVNd 2023-10-12T02:01:28.691 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0666 /tmp/tmp.6ovGwPOVNd 2023-10-12T02:01:28.835 DEBUG:teuthology.orchestra.remote:smithi125:/tmp/tmp.6ovGwPOVNd is 6KB 2023-10-12T02:01:28.883 DEBUG:teuthology.orchestra.run.smithi125:> rm -fr /tmp/tmp.6ovGwPOVNd 2023-10-12T02:01:28.897 DEBUG:teuthology.orchestra.run.smithi125:> sudo /bin/ls /boot/loader/entries || true 2023-10-12T02:01:28.969 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-12T02:01:28.969 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-12T02:01:28.969 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-12T02:01:28.969 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-375.el9.x86_64.conf 2023-10-12T02:01:28.970 DEBUG:teuthology.orchestra.run.smithi125:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-375.el9.x86_64 2023-10-12T02:01:29.131 DEBUG:teuthology.orchestra.run.smithi125:> sudo shutdown -r now 2023-10-12T02:01:58.478 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2023-10-12T02:01:58.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:01:58.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:01:59.135 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-12T02:01:59.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-12T02:01:59.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:02:16.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-12T02:02:25.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:02:25.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:02:28.897 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-12T02:02:40.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:02:40.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:02:43.962 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-12T02:02:58.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:02:58.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:02:59.139 DEBUG:teuthology.orchestra.remote:timed out 2023-10-12T02:03:08.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-12T02:03:08.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:03:08.598 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-10-12T02:03:08.968 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-12T02:03:08.969 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-375.el9.x86_64"... 2023-10-12T02:03:08.969 DEBUG:teuthology.orchestra.run.smithi125:> uname -r 2023-10-12T02:03:08.983 INFO:teuthology.orchestra.run.smithi125.stdout:5.14.0-375.el9.x86_64 2023-10-12T02:03:08.983 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-375.el9.x86_64 vs 5.14.0-375.el9.x86_64 2023-10-12T02:03:08.984 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-12T02:03:08.984 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-12T02:03:09.985 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-12T02:03:09.986 DEBUG:teuthology.orchestra.run.smithi125:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-12T02:03:10.058 INFO:teuthology.orchestra.run.smithi125.stdout:ttyS1 2023-10-12T02:03:10.099 DEBUG:teuthology.parallel:result is None 2023-10-12T02:03:17.508 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-10-12T02:03:35.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:03:35.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:03:35.846 DEBUG:teuthology.orchestra.run.smithi158:> true 2023-10-12T02:03:36.201 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:03:36.202 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-375.el9.x86_64"... 2023-10-12T02:03:36.202 DEBUG:teuthology.orchestra.run.smithi158:> uname -r 2023-10-12T02:03:36.217 INFO:teuthology.orchestra.run.smithi158.stdout:5.14.0-375.el9.x86_64 2023-10-12T02:03:36.217 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-375.el9.x86_64 vs 5.14.0-375.el9.x86_64 2023-10-12T02:03:36.217 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-12T02:03:36.218 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-12T02:03:37.218 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2023-10-12T02:03:37.219 DEBUG:teuthology.orchestra.run.smithi158:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-12T02:03:37.309 INFO:teuthology.orchestra.run.smithi158.stdout:ttyS1 2023-10-12T02:03:37.358 DEBUG:teuthology.parallel:result is None 2023-10-12T02:03:37.359 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-12T02:03:37.366 INFO:teuthology.task.internal:Creating test directory... 2023-10-12T02:03:37.367 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-12T02:03:37.370 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-12T02:03:37.394 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-12T02:03:37.401 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-12T02:03:37.413 INFO:teuthology.task.internal:Creating archive directory... 2023-10-12T02:03:37.414 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-12T02:03:37.428 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-12T02:03:37.501 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-12T02:03:37.508 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-12T02:03:37.508 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-12T02:03:37.511 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-12T02:03:37.546 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-12T02:03:37.559 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-12T02:03:37.562 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-12T02:03:37.586 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-12T02:03:37.588 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-12T02:03:37.595 INFO:teuthology.task.internal:Configuring sudo... 2023-10-12T02:03:37.596 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-12T02:03:37.605 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-12T02:03:37.648 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-12T02:03:37.657 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-12T02:03:37.658 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-12T02:03:37.674 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-12T02:03:37.699 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-12T02:03:37.768 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-12T02:03:37.854 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-12T02:03:37.894 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-12T02:03:37.937 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:03:37.938 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-12T02:03:38.004 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-12T02:03:38.047 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-12T02:03:38.226 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-12T02:03:38.267 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-12T02:03:38.309 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:03:38.310 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-12T02:03:38.376 DEBUG:teuthology.orchestra.run.smithi125:> sudo service rsyslog restart 2023-10-12T02:03:38.379 DEBUG:teuthology.orchestra.run.smithi158:> sudo service rsyslog restart 2023-10-12T02:03:38.423 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-12T02:03:38.516 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-12T02:03:38.784 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-12T02:03:38.793 INFO:teuthology.task.internal:Starting timer... 2023-10-12T02:03:38.794 INFO:teuthology.run_tasks:Running task pcp... 2023-10-12T02:03:38.804 INFO:teuthology.run_tasks:Running task selinux... 2023-10-12T02:03:38.898 DEBUG:teuthology.orchestra.run.smithi125:> sudo service auditd rotate 2023-10-12T02:03:38.959 INFO:teuthology.orchestra.run.smithi125.stdout:Rotating logs: 2023-10-12T02:03:38.962 DEBUG:teuthology.orchestra.run.smithi158:> sudo service auditd rotate 2023-10-12T02:03:39.024 INFO:teuthology.orchestra.run.smithi158.stdout:Rotating logs: 2023-10-12T02:03:39.026 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-12T02:03:39.026 DEBUG:teuthology.orchestra.run.smithi125:> /usr/sbin/getenforce 2023-10-12T02:03:39.058 INFO:teuthology.orchestra.run.smithi125.stdout:Permissive 2023-10-12T02:03:39.059 DEBUG:teuthology.orchestra.run.smithi158:> /usr/sbin/getenforce 2023-10-12T02:03:39.081 INFO:teuthology.orchestra.run.smithi158.stdout:Permissive 2023-10-12T02:03:39.081 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi125.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi158.front.sepia.ceph.com': 'permissive'} 2023-10-12T02:03:39.082 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-12T02:03:39.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T02:03:39.127 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-12T02:03:39.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T02:03:39.154 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-12T02:03:39.155 DEBUG:teuthology.orchestra.run.smithi125:> sudo /usr/sbin/setenforce permissive 2023-10-12T02:03:39.202 DEBUG:teuthology.orchestra.run.smithi158:> sudo /usr/sbin/setenforce permissive 2023-10-12T02:03:39.242 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-12T02:03:39.252 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-12T02:03:39.257 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-12T02:03:39.376 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-12T02:03:39.393 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-12T02:03:39.394 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi125.front.sepia.ceph.com,smithi158.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-12T02:09:33.384 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi125.front.sepia.ceph.com'), Remote(name='ubuntu@smithi158.front.sepia.ceph.com')] 2023-10-12T02:09:33.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-12T02:09:33.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:09:33.474 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-10-12T02:09:33.541 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-12T02:09:33.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:09:33.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-12T02:09:33.625 DEBUG:teuthology.orchestra.run.smithi158:> true 2023-10-12T02:09:33.696 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-10-12T02:09:33.697 INFO:teuthology.run_tasks:Running task clock... 2023-10-12T02:09:33.707 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-12T02:09:33.708 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-12T02:09:33.709 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-12T02:09:33.712 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-12T02:09:33.713 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-12T02:09:33.743 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-12T02:09:33.760 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-12T02:09:33.767 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-12T02:09:33.784 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-12T02:09:33.790 INFO:teuthology.orchestra.run.smithi125.stderr:sudo: ntpd: command not found 2023-10-12T02:09:33.804 INFO:teuthology.orchestra.run.smithi125.stdout:506 Cannot talk to daemon 2023-10-12T02:09:33.813 INFO:teuthology.orchestra.run.smithi158.stderr:sudo: ntpd: command not found 2023-10-12T02:09:33.821 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-12T02:09:33.826 INFO:teuthology.orchestra.run.smithi158.stdout:506 Cannot talk to daemon 2023-10-12T02:09:33.837 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-12T02:09:33.842 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-12T02:09:33.858 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-12T02:09:33.880 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2023-10-12T02:09:33.884 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-12T02:09:33.884 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2023-10-12T02:09:33.884 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.884 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.884 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.885 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.902 INFO:teuthology.orchestra.run.smithi158.stderr:bash: line 1: ntpq: command not found 2023-10-12T02:09:33.905 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-12T02:09:33.906 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2023-10-12T02:09:33.906 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.906 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.906 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.906 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T02:09:33.907 INFO:teuthology.run_tasks:Running task install... 2023-10-12T02:09:33.917 DEBUG:teuthology.task.install:project ceph 2023-10-12T02:09:33.917 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5'}} 2023-10-12T02:09:33.917 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5'} 2023-10-12T02:09:33.918 INFO:teuthology.task.install:Using flavor: crimson 2023-10-12T02:09:33.923 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-12T02:09:33.924 INFO:teuthology.task.install:extra packages: [] 2023-10-12T02:09:33.925 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5', 'tag': None, 'wait_for_package': False} 2023-10-12T02:09:33.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 2023-10-12T02:09:33.927 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5', 'tag': None, 'wait_for_package': False} 2023-10-12T02:09:33.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 2023-10-12T02:09:34.321 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/ 2023-10-12T02:09:34.321 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6684.g9587ffdf 2023-10-12T02:09:34.433 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-12T02:09:34.433 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:09:34.434 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-12T02:09:34.463 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-10-12T02:09:34.463 DEBUG:teuthology.orchestra.run.smithi158:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-12T02:09:34.532 DEBUG:teuthology.orchestra.run.smithi158:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-12T02:09:34.611 DEBUG:teuthology.orchestra.run.smithi158:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-12T02:09:34.677 INFO:teuthology.orchestra.run.smithi158.stdout:check_obsoletes = 1 2023-10-12T02:09:34.680 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum clean all 2023-10-12T02:09:34.918 INFO:teuthology.orchestra.run.smithi158.stdout:70 files removed 2023-10-12T02:09:34.945 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-radosgw 2023-10-12T02:09:35.252 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/ 2023-10-12T02:09:35.252 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6684.g9587ffdf 2023-10-12T02:09:35.426 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yingxin-crimson-osd-crosscore-pg-submission/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-12T02:09:35.426 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:09:35.427 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-12T02:09:35.459 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-10-12T02:09:35.460 DEBUG:teuthology.orchestra.run.smithi125:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-12T02:09:35.530 DEBUG:teuthology.orchestra.run.smithi125:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-12T02:09:35.613 DEBUG:teuthology.orchestra.run.smithi125:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-12T02:09:35.674 INFO:teuthology.orchestra.run.smithi158.stdout:ceph packages for x86_64 197 kB/s | 81 kB 00:00 2023-10-12T02:09:35.681 INFO:teuthology.orchestra.run.smithi125.stdout:check_obsoletes = 1 2023-10-12T02:09:35.683 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum clean all 2023-10-12T02:09:35.921 INFO:teuthology.orchestra.run.smithi125.stdout:70 files removed 2023-10-12T02:09:35.948 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-radosgw 2023-10-12T02:09:36.157 INFO:teuthology.orchestra.run.smithi158.stdout:ceph noarch packages 22 kB/s | 9.4 kB 00:00 2023-10-12T02:09:36.658 INFO:teuthology.orchestra.run.smithi158.stdout:ceph source packages 4.2 kB/s | 2.0 kB 00:00 2023-10-12T02:09:36.777 INFO:teuthology.orchestra.run.smithi125.stdout:ceph packages for x86_64 158 kB/s | 81 kB 00:00 2023-10-12T02:09:37.111 INFO:teuthology.orchestra.run.smithi125.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2023-10-12T02:09:37.269 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-12T02:09:37.528 INFO:teuthology.orchestra.run.smithi125.stdout:ceph source packages 5.2 kB/s | 2.0 kB 00:00 2023-10-12T02:09:37.947 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - AppStream 16 MB/s | 6.3 MB 00:00 2023-10-12T02:09:38.703 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2023-10-12T02:09:39.281 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2023-10-12T02:09:41.197 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 7.9 MB 00:01 2023-10-12T02:09:42.760 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 7.9 MB 00:03 2023-10-12T02:09:43.531 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - AppStream 14 MB/s | 18 MB 00:01 2023-10-12T02:09:45.636 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 18 MB 00:01 2023-10-12T02:09:48.898 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 5.7 MB 00:02 2023-10-12T02:09:50.815 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 14 kB 00:00 2023-10-12T02:09:51.870 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 5.7 MB 00:02 2023-10-12T02:09:52.767 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 10 MB/s | 19 MB 00:01 2023-10-12T02:09:54.097 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 14 kB 00:00 2023-10-12T02:09:56.381 INFO:teuthology.orchestra.run.smithi158.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 19 MB 00:02 2023-10-12T02:09:57.217 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2023-10-12T02:09:59.558 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:09:59.571 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:09:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:09:59.572 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:09:59.572 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:09:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 8.4 M 2023-10-12T02:09:59.572 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrading: 2023-10-12T02:09:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: librados2 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 3.1 M 2023-10-12T02:09:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 3.0 M 2023-10-12T02:09:59.573 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:09:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options 2023-10-12T02:09:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-12T02:09:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 6.6 M 2023-10-12T02:09:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 17 M 2023-10-12T02:09:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 24 k 2023-10-12T02:09:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-12T02:09:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-12T02:09:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-12T02:09:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-12T02:09:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 727 k 2023-10-12T02:09:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-12T02:09:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-12T02:09:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-12T02:09:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 469 k 2023-10-12T02:09:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-12T02:09:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 4.5 M 2023-10-12T02:09:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-12T02:09:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-12T02:09:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-12T02:09:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse 2023-10-12T02:09:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 45 k 2023-10-12T02:09:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 127 k 2023-10-12T02:09:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 162 k 2023-10-12T02:09:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-12T02:09:59.579 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 321 k 2023-10-12T02:09:59.579 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 298 k 2023-10-12T02:09:59.579 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 99 k 2023-10-12T02:09:59.580 INFO:teuthology.orchestra.run.smithi125.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-12T02:09:59.580 INFO:teuthology.orchestra.run.smithi125.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-12T02:09:59.580 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:09:59.581 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:09:59.581 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:09:59.582 INFO:teuthology.orchestra.run.smithi125.stdout:Install 28 Packages 2023-10-12T02:09:59.582 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrade 2 Packages 2023-10-12T02:09:59.582 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:09:59.583 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 54 M 2023-10-12T02:09:59.584 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:10:00.765 INFO:teuthology.orchestra.run.smithi158.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2023-10-12T02:10:01.959 INFO:teuthology.orchestra.run.smithi125.stdout:(1/30): ceph-base-18.0.0-6684.g9587ffdf.el9.x86 3.4 MB/s | 6.6 MB 00:01 2023-10-12T02:10:02.068 INFO:teuthology.orchestra.run.smithi125.stdout:(2/30): ceph-selinux-18.0.0-6684.g9587ffdf.el9. 222 kB/s | 24 kB 00:00 2023-10-12T02:10:02.384 INFO:teuthology.orchestra.run.smithi125.stdout:(3/30): libcephfs2-18.0.0-6684.g9587ffdf.el9.x8 2.2 MB/s | 727 kB 00:00 2023-10-12T02:10:02.544 INFO:teuthology.orchestra.run.smithi125.stdout:(4/30): libradosstriper1-18.0.0-6684.g9587ffdf. 2.9 MB/s | 469 kB 00:00 2023-10-12T02:10:02.803 INFO:teuthology.orchestra.run.smithi125.stdout:(5/30): ceph-radosgw-18.0.0-6684.g9587ffdf.el9. 3.0 MB/s | 8.4 MB 00:02 2023-10-12T02:10:02.910 INFO:teuthology.orchestra.run.smithi125.stdout:(6/30): python3-ceph-argparse-18.0.0-6684.g9587 418 kB/s | 45 kB 00:00 2023-10-12T02:10:02.986 INFO:teuthology.orchestra.run.smithi125.stdout:(7/30): python3-ceph-common-18.0.0-6684.g9587ff 1.6 MB/s | 127 kB 00:00 2023-10-12T02:10:03.078 INFO:teuthology.orchestra.run.smithi125.stdout:(8/30): python3-cephfs-18.0.0-6684.g9587ffdf.el 1.7 MB/s | 162 kB 00:00 2023-10-12T02:10:03.147 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:10:03.165 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:10:03.166 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:10:03.166 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:10:03.166 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:10:03.166 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 8.4 M 2023-10-12T02:10:03.166 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrading: 2023-10-12T02:10:03.167 INFO:teuthology.orchestra.run.smithi158.stdout: librados2 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 3.1 M 2023-10-12T02:10:03.167 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 3.0 M 2023-10-12T02:10:03.167 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:10:03.168 INFO:teuthology.orchestra.run.smithi158.stdout: boost-program-options 2023-10-12T02:10:03.168 INFO:teuthology.orchestra.run.smithi158.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-12T02:10:03.168 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 6.6 M 2023-10-12T02:10:03.168 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 17 M 2023-10-12T02:10:03.168 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 24 k 2023-10-12T02:10:03.169 INFO:teuthology.orchestra.run.smithi158.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-12T02:10:03.169 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-12T02:10:03.169 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-12T02:10:03.169 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-12T02:10:03.169 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 727 k 2023-10-12T02:10:03.170 INFO:teuthology.orchestra.run.smithi158.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-12T02:10:03.170 INFO:teuthology.orchestra.run.smithi158.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-12T02:10:03.170 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-12T02:10:03.170 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 469 k 2023-10-12T02:10:03.170 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-12T02:10:03.171 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2 x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 4.5 M 2023-10-12T02:10:03.171 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-12T02:10:03.171 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-12T02:10:03.171 INFO:teuthology.orchestra.run.smithi158.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-12T02:10:03.171 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse 2023-10-12T02:10:03.172 INFO:teuthology.orchestra.run.smithi158.stdout: x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 45 k 2023-10-12T02:10:03.172 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 127 k 2023-10-12T02:10:03.172 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 162 k 2023-10-12T02:10:03.172 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-12T02:10:03.172 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 321 k 2023-10-12T02:10:03.173 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 298 k 2023-10-12T02:10:03.173 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 99 k 2023-10-12T02:10:03.173 INFO:teuthology.orchestra.run.smithi158.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-12T02:10:03.173 INFO:teuthology.orchestra.run.smithi158.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-12T02:10:03.173 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:03.174 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:10:03.174 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:10:03.174 INFO:teuthology.orchestra.run.smithi158.stdout:Install 28 Packages 2023-10-12T02:10:03.174 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrade 2 Packages 2023-10-12T02:10:03.175 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:03.175 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 54 M 2023-10-12T02:10:03.176 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:10:03.220 INFO:teuthology.orchestra.run.smithi125.stdout:(9/30): librgw2-18.0.0-6684.g9587ffdf.el9.x86_6 6.7 MB/s | 4.5 MB 00:00 2023-10-12T02:10:03.311 INFO:teuthology.orchestra.run.smithi125.stdout:(10/30): python3-rbd-18.0.0-6684.g9587ffdf.el9. 3.2 MB/s | 298 kB 00:00 2023-10-12T02:10:03.370 INFO:teuthology.orchestra.run.smithi125.stdout:(11/30): python3-rgw-18.0.0-6684.g9587ffdf.el9. 1.7 MB/s | 99 kB 00:00 2023-10-12T02:10:03.478 INFO:teuthology.orchestra.run.smithi125.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 977 kB/s | 106 kB 00:00 2023-10-12T02:10:03.504 INFO:teuthology.orchestra.run.smithi125.stdout:(13/30): python3-rados-18.0.0-6684.g9587ffdf.el 753 kB/s | 321 kB 00:00 2023-10-12T02:10:03.529 INFO:teuthology.orchestra.run.smithi125.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-10-12T02:10:03.571 INFO:teuthology.orchestra.run.smithi125.stdout:(15/30): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2023-10-12T02:10:03.596 INFO:teuthology.orchestra.run.smithi125.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-10-12T02:10:03.622 INFO:teuthology.orchestra.run.smithi125.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 828 kB/s | 41 kB 00:00 2023-10-12T02:10:03.714 INFO:teuthology.orchestra.run.smithi125.stdout:(18/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2023-10-12T02:10:04.055 INFO:teuthology.orchestra.run.smithi125.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 255 kB/s | 111 kB 00:00 2023-10-12T02:10:04.264 INFO:teuthology.orchestra.run.smithi125.stdout:(20/30): ceph-common-18.0.0-6684.g9587ffdf.el9. 4.0 MB/s | 17 MB 00:04 2023-10-12T02:10:04.373 INFO:teuthology.orchestra.run.smithi125.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 469 kB/s | 309 kB 00:00 2023-10-12T02:10:04.390 INFO:teuthology.orchestra.run.smithi158.stdout:(1/30): ceph-base-18.0.0-6684.g9587ffdf.el9.x86 6.9 MB/s | 6.6 MB 00:00 2023-10-12T02:10:04.458 INFO:teuthology.orchestra.run.smithi158.stdout:(2/30): ceph-selinux-18.0.0-6684.g9587ffdf.el9. 359 kB/s | 24 kB 00:00 2023-10-12T02:10:04.506 INFO:teuthology.orchestra.run.smithi125.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 365 kB/s | 49 kB 00:00 2023-10-12T02:10:04.581 INFO:teuthology.orchestra.run.smithi125.stdout:(23/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 77 kB/s | 24 kB 00:00 2023-10-12T02:10:04.608 INFO:teuthology.orchestra.run.smithi158.stdout:(3/30): libcephfs2-18.0.0-6684.g9587ffdf.el9.x8 4.7 MB/s | 727 kB 00:00 2023-10-12T02:10:04.651 INFO:teuthology.orchestra.run.smithi125.stdout:(24/30): libarrow-9.0.0-9.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2023-10-12T02:10:04.673 INFO:teuthology.orchestra.run.smithi125.stdout:(25/30): libunwind-1.6.2-1.el9.x86_64.rpm 403 kB/s | 67 kB 00:00 2023-10-12T02:10:04.692 INFO:teuthology.orchestra.run.smithi158.stdout:(4/30): libradosstriper1-18.0.0-6684.g9587ffdf. 5.5 MB/s | 469 kB 00:00 2023-10-12T02:10:04.790 INFO:teuthology.orchestra.run.smithi125.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2023-10-12T02:10:05.015 INFO:teuthology.orchestra.run.smithi125.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2023-10-12T02:10:05.116 INFO:teuthology.orchestra.run.smithi125.stdout:(28/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.5 MB/s | 837 kB 00:00 2023-10-12T02:10:05.184 INFO:teuthology.orchestra.run.smithi158.stdout:(5/30): librgw2-18.0.0-6684.g9587ffdf.el9.x86_6 9.1 MB/s | 4.5 MB 00:00 2023-10-12T02:10:05.276 INFO:teuthology.orchestra.run.smithi158.stdout:(6/30): python3-ceph-argparse-18.0.0-6684.g9587 493 kB/s | 45 kB 00:00 2023-10-12T02:10:05.426 INFO:teuthology.orchestra.run.smithi158.stdout:(7/30): ceph-radosgw-18.0.0-6684.g9587ffdf.el9. 4.2 MB/s | 8.4 MB 00:01 2023-10-12T02:10:05.451 INFO:teuthology.orchestra.run.smithi158.stdout:(8/30): python3-ceph-common-18.0.0-6684.g9587ff 724 kB/s | 127 kB 00:00 2023-10-12T02:10:05.541 INFO:teuthology.orchestra.run.smithi125.stdout:(29/30): librados2-18.0.0-6684.g9587ffdf.el9.x8 4.1 MB/s | 3.1 MB 00:00 2023-10-12T02:10:05.626 INFO:teuthology.orchestra.run.smithi158.stdout:(9/30): ceph-common-18.0.0-6684.g9587ffdf.el9.x 7.9 MB/s | 17 MB 00:02 2023-10-12T02:10:05.652 INFO:teuthology.orchestra.run.smithi158.stdout:(10/30): python3-cephfs-18.0.0-6684.g9587ffdf.e 718 kB/s | 162 kB 00:00 2023-10-12T02:10:05.677 INFO:teuthology.orchestra.run.smithi158.stdout:(11/30): python3-rados-18.0.0-6684.g9587ffdf.el 1.4 MB/s | 321 kB 00:00 2023-10-12T02:10:05.702 INFO:teuthology.orchestra.run.smithi158.stdout:(12/30): python3-rbd-18.0.0-6684.g9587ffdf.el9. 3.9 MB/s | 298 kB 00:00 2023-10-12T02:10:05.778 INFO:teuthology.orchestra.run.smithi158.stdout:(13/30): python3-rgw-18.0.0-6684.g9587ffdf.el9. 791 kB/s | 99 kB 00:00 2023-10-12T02:10:05.783 INFO:teuthology.orchestra.run.smithi125.stdout:(30/30): librbd1-18.0.0-6684.g9587ffdf.el9.x86_ 4.0 MB/s | 3.0 MB 00:00 2023-10-12T02:10:05.787 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:10:05.787 INFO:teuthology.orchestra.run.smithi125.stdout:Total 8.7 MB/s | 54 MB 00:06 2023-10-12T02:10:05.799 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:10:05.828 INFO:teuthology.orchestra.run.smithi158.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 703 kB/s | 106 kB 00:00 2023-10-12T02:10:05.836 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:10:05.836 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:10:05.853 INFO:teuthology.orchestra.run.smithi158.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2023-10-12T02:10:05.881 INFO:teuthology.orchestra.run.smithi158.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2023-10-12T02:10:05.904 INFO:teuthology.orchestra.run.smithi158.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2023-10-12T02:10:05.929 INFO:teuthology.orchestra.run.smithi158.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 549 kB/s | 41 kB 00:00 2023-10-12T02:10:06.071 INFO:teuthology.orchestra.run.smithi158.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 663 kB/s | 111 kB 00:00 2023-10-12T02:10:06.130 INFO:teuthology.orchestra.run.smithi158.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.5 MB/s | 309 kB 00:00 2023-10-12T02:10:06.172 INFO:teuthology.orchestra.run.smithi158.stdout:(21/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 582 kB/s | 24 kB 00:00 2023-10-12T02:10:06.230 INFO:teuthology.orchestra.run.smithi158.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2023-10-12T02:10:06.298 INFO:teuthology.orchestra.run.smithi158.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2023-10-12T02:10:06.349 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:10:06.349 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:10:06.531 INFO:teuthology.orchestra.run.smithi158.stdout:(24/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.5 MB/s | 837 kB 00:00 2023-10-12T02:10:06.623 INFO:teuthology.orchestra.run.smithi158.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2023-10-12T02:10:06.648 INFO:teuthology.orchestra.run.smithi158.stdout:(26/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 61 kB/s | 45 kB 00:00 2023-10-12T02:10:06.807 INFO:teuthology.orchestra.run.smithi158.stdout:(27/30): libarrow-9.0.0-9.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2023-10-12T02:10:07.057 INFO:teuthology.orchestra.run.smithi158.stdout:(28/30): librados2-18.0.0-6684.g9587ffdf.el9.x8 7.6 MB/s | 3.1 MB 00:00 2023-10-12T02:10:07.200 INFO:teuthology.orchestra.run.smithi158.stdout:(29/30): librbd1-18.0.0-6684.g9587ffdf.el9.x86_ 7.7 MB/s | 3.0 MB 00:00 2023-10-12T02:10:07.242 INFO:teuthology.orchestra.run.smithi158.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2023-10-12T02:10:07.246 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:10:07.246 INFO:teuthology.orchestra.run.smithi158.stdout:Total 13 MB/s | 54 MB 00:04 2023-10-12T02:10:07.258 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:10:07.298 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:10:07.298 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:10:07.776 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:10:07.844 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:10:07.845 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:10:08.127 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-10-12T02:10:08.202 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/32 2023-10-12T02:10:08.277 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/32 2023-10-12T02:10:08.373 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 3/32 2023-10-12T02:10:08.395 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/32 2023-10-12T02:10:08.458 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/32 2023-10-12T02:10:08.504 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/32 2023-10-12T02:10:08.533 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/32 2023-10-12T02:10:08.562 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : liboath-2.6.7-2.el9.x86_64 7/32 2023-10-12T02:10:08.614 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-argparse-2:18.0.0-6684.g9587ffdf.el9. 8/32 2023-10-12T02:10:08.691 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 9/32 2023-10-12T02:10:08.715 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_6 10/32 2023-10-12T02:10:08.802 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_6 10/32 2023-10-12T02:10:08.847 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : thrift-0.15.0-2.el9.x86_64 11/32 2023-10-12T02:10:08.956 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-10-12T02:10:08.991 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-10-12T02:10:09.031 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-10-12T02:10:09.064 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-10-12T02:10:09.091 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-10-12T02:10:09.128 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-10-12T02:10:09.160 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-12T02:10:09.232 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:10:09.486 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-10-12T02:10:09.515 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 20/32 2023-10-12T02:10:09.575 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-10-12T02:10:09.597 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 20/32 2023-10-12T02:10:09.630 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 21/32 2023-10-12T02:10:09.672 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-10-12T02:10:09.727 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/32 2023-10-12T02:10:09.803 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/32 2023-10-12T02:10:09.906 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 3/32 2023-10-12T02:10:09.929 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/32 2023-10-12T02:10:09.992 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/32 2023-10-12T02:10:10.036 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/32 2023-10-12T02:10:10.075 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/32 2023-10-12T02:10:10.113 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : liboath-2.6.7-2.el9.x86_64 7/32 2023-10-12T02:10:10.119 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-10-12T02:10:10.142 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 24/32 2023-10-12T02:10:10.157 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-argparse-2:18.0.0-6684.g9587ffdf.el9. 8/32 2023-10-12T02:10:10.229 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 9/32 2023-10-12T02:10:10.238 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 24/32 2023-10-12T02:10:10.258 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_6 10/32 2023-10-12T02:10:10.305 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 25/32 2023-10-12T02:10:10.336 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-common-2:18.0.0-6684.g9587ffdf.el9.x8 26/32 2023-10-12T02:10:10.348 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_6 10/32 2023-10-12T02:10:10.390 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : thrift-0.15.0-2.el9.x86_64 11/32 2023-10-12T02:10:10.498 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-10-12T02:10:10.527 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-10-12T02:10:10.568 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-10-12T02:10:10.609 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-10-12T02:10:10.636 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-10-12T02:10:10.671 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-10-12T02:10:10.704 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-12T02:10:11.034 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-10-12T02:10:11.068 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 20/32 2023-10-12T02:10:11.182 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 20/32 2023-10-12T02:10:11.214 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 21/32 2023-10-12T02:10:11.251 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-10-12T02:10:11.681 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-10-12T02:10:11.713 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 24/32 2023-10-12T02:10:11.775 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 24/32 2023-10-12T02:10:11.851 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 25/32 2023-10-12T02:10:11.882 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-common-2:18.0.0-6684.g9587ffdf.el9.x8 26/32 2023-10-12T02:10:12.368 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 27/32 2023-10-12T02:10:12.399 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 27/32 2023-10-12T02:10:13.138 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 27/32 2023-10-12T02:10:13.185 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 28/32 2023-10-12T02:10:13.409 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 28/32 2023-10-12T02:10:13.409 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-12T02:10:13.409 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-12T02:10:13.410 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:10:13.445 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/32 2023-10-12T02:10:14.003 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 27/32 2023-10-12T02:10:14.043 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 27/32 2023-10-12T02:10:14.783 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 27/32 2023-10-12T02:10:14.822 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 28/32 2023-10-12T02:10:14.971 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 28/32 2023-10-12T02:10:14.972 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-12T02:10:14.972 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-12T02:10:14.972 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:15.007 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/32 2023-10-12T02:10:23.045 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/32 2023-10-12T02:10:23.045 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /sys 2023-10-12T02:10:23.045 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /proc 2023-10-12T02:10:23.046 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /mnt 2023-10-12T02:10:23.046 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /var/tmp 2023-10-12T02:10:23.046 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /home 2023-10-12T02:10:23.046 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /root 2023-10-12T02:10:23.047 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /tmp 2023-10-12T02:10:23.047 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:10:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/32 2023-10-12T02:10:23.926 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/32 2023-10-12T02:10:23.926 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:10:23.926 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-12T02:10:23.926 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-12T02:10:23.927 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-12T02:10:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:10:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-12T02:10:24.012 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-12T02:10:24.013 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-12T02:10:24.472 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/32 2023-10-12T02:10:24.472 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /sys 2023-10-12T02:10:24.472 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /proc 2023-10-12T02:10:24.473 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /mnt 2023-10-12T02:10:24.473 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /var/tmp 2023-10-12T02:10:24.473 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /home 2023-10-12T02:10:24.473 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /root 2023-10-12T02:10:24.473 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /tmp 2023-10-12T02:10:24.474 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:25.100 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-12T02:10:25.101 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/32 2023-10-12T02:10:25.101 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/32 2023-10-12T02:10:25.101 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 3/32 2023-10-12T02:10:25.101 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/32 2023-10-12T02:10:25.101 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 5/32 2023-10-12T02:10:25.102 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_6 6/32 2023-10-12T02:10:25.103 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 7/32 2023-10-12T02:10:25.103 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6684.g9587ffdf.el9. 8/32 2023-10-12T02:10:25.103 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-common-2:18.0.0-6684.g9587ffdf.el9.x8 9/32 2023-10-12T02:10:25.104 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 10/32 2023-10-12T02:10:25.104 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 11/32 2023-10-12T02:10:25.104 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 12/32 2023-10-12T02:10:25.105 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 13/32 2023-10-12T02:10:25.106 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-10-12T02:10:25.106 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-10-12T02:10:25.106 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-10-12T02:10:25.107 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-10-12T02:10:25.107 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-12T02:10:25.107 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-10-12T02:10:25.107 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-10-12T02:10:25.107 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-10-12T02:10:25.108 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-10-12T02:10:25.108 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-10-12T02:10:25.108 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-10-12T02:10:25.108 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-10-12T02:10:25.108 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-10-12T02:10:25.108 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-10-12T02:10:25.109 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-10-12T02:10:25.109 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/32 2023-10-12T02:10:25.109 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-10-12T02:10:25.109 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 31/32 2023-10-12T02:10:25.332 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/32 2023-10-12T02:10:25.354 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/32 2023-10-12T02:10:25.355 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:10:25.355 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-12T02:10:25.355 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-12T02:10:25.356 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-12T02:10:25.356 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:25.357 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-12T02:10:25.442 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-12T02:10:25.442 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-12T02:10:26.214 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-10-12T02:10:26.214 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:10:26.214 INFO:teuthology.orchestra.run.smithi125.stdout:Upgraded: 2023-10-12T02:10:26.215 INFO:teuthology.orchestra.run.smithi125.stdout: librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.215 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.215 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:10:26.215 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-12T02:10:26.216 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.216 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.216 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.216 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.216 INFO:teuthology.orchestra.run.smithi125.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-12T02:10:26.217 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-12T02:10:26.217 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-12T02:10:26.217 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-12T02:10:26.217 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.218 INFO:teuthology.orchestra.run.smithi125.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-12T02:10:26.218 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-12T02:10:26.218 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-12T02:10:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-12T02:10:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-12T02:10:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-12T02:10:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-12T02:10:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.223 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.223 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-12T02:10:26.223 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.223 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.224 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:26.224 INFO:teuthology.orchestra.run.smithi125.stdout: re2-1:20211101-3.el9.x86_64 2023-10-12T02:10:26.224 INFO:teuthology.orchestra.run.smithi125.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-12T02:10:26.224 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:10:26.225 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:10:26.466 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-test 2023-10-12T02:10:26.624 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-12T02:10:26.625 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/32 2023-10-12T02:10:26.625 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/32 2023-10-12T02:10:26.625 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 3/32 2023-10-12T02:10:26.625 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/32 2023-10-12T02:10:26.625 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 5/32 2023-10-12T02:10:26.626 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_6 6/32 2023-10-12T02:10:26.626 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 7/32 2023-10-12T02:10:26.626 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6684.g9587ffdf.el9. 8/32 2023-10-12T02:10:26.626 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-common-2:18.0.0-6684.g9587ffdf.el9.x8 9/32 2023-10-12T02:10:26.627 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 10/32 2023-10-12T02:10:26.627 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 11/32 2023-10-12T02:10:26.627 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 12/32 2023-10-12T02:10:26.627 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 13/32 2023-10-12T02:10:26.627 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-10-12T02:10:26.628 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-10-12T02:10:26.628 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-10-12T02:10:26.628 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-10-12T02:10:26.628 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-12T02:10:26.629 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-10-12T02:10:26.629 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-10-12T02:10:26.629 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-10-12T02:10:26.629 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-10-12T02:10:26.630 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-10-12T02:10:26.630 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-10-12T02:10:26.630 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-10-12T02:10:26.630 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-10-12T02:10:26.631 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-10-12T02:10:26.631 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-10-12T02:10:26.632 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/32 2023-10-12T02:10:26.632 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-10-12T02:10:26.632 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 31/32 2023-10-12T02:10:27.039 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:00:30 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:10:27.179 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:10:27.180 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:10:27.181 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:10:27.181 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:10:27.181 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:10:27.181 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 117 M 2023-10-12T02:10:27.181 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:10:27.182 INFO:teuthology.orchestra.run.smithi125.stdout: cryptopp x86_64 8.6.0-1.el9 epel 1.3 M 2023-10-12T02:10:27.182 INFO:teuthology.orchestra.run.smithi125.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-12T02:10:27.182 INFO:teuthology.orchestra.run.smithi125.stdout: libasan x86_64 11.4.1-2.3.el9 CentOS-AppStream 418 k 2023-10-12T02:10:27.182 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 161 k 2023-10-12T02:10:27.182 INFO:teuthology.orchestra.run.smithi125.stdout: libubsan x86_64 11.4.1-2.3.el9 CentOS-AppStream 186 k 2023-10-12T02:10:27.183 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-12T02:10:27.183 INFO:teuthology.orchestra.run.smithi125.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-12T02:10:27.183 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-12T02:10:27.183 INFO:teuthology.orchestra.run.smithi125.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2023-10-12T02:10:27.183 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:10:27.184 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:10:27.184 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:10:27.184 INFO:teuthology.orchestra.run.smithi125.stdout:Install 10 Packages 2023-10-12T02:10:27.184 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:10:27.185 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 120 M 2023-10-12T02:10:27.185 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 1.1 G 2023-10-12T02:10:27.185 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:10:27.457 INFO:teuthology.orchestra.run.smithi125.stdout:(1/10): libasan-11.4.1-2.3.el9.x86_64.rpm 2.7 MB/s | 418 kB 00:00 2023-10-12T02:10:27.499 INFO:teuthology.orchestra.run.smithi125.stdout:(2/10): libubsan-11.4.1-2.3.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2023-10-12T02:10:27.541 INFO:teuthology.orchestra.run.smithi125.stdout:(3/10): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.2 MB/s | 217 kB 00:00 2023-10-12T02:10:27.583 INFO:teuthology.orchestra.run.smithi125.stdout:(4/10): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2023-10-12T02:10:27.616 INFO:teuthology.orchestra.run.smithi125.stdout:(5/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2023-10-12T02:10:27.696 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-10-12T02:10:27.697 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:27.697 INFO:teuthology.orchestra.run.smithi158.stdout:Upgraded: 2023-10-12T02:10:27.697 INFO:teuthology.orchestra.run.smithi158.stdout: librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.697 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.697 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:10:27.698 INFO:teuthology.orchestra.run.smithi158.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-12T02:10:27.698 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.698 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.698 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.698 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.699 INFO:teuthology.orchestra.run.smithi158.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-12T02:10:27.699 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-12T02:10:27.699 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-12T02:10:27.699 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-12T02:10:27.699 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.700 INFO:teuthology.orchestra.run.smithi158.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-12T02:10:27.700 INFO:teuthology.orchestra.run.smithi158.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-12T02:10:27.700 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-12T02:10:27.700 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.700 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-12T02:10:27.701 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.701 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-12T02:10:27.701 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-12T02:10:27.701 INFO:teuthology.orchestra.run.smithi158.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-12T02:10:27.701 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.702 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.702 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.702 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-12T02:10:27.702 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.702 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.703 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:10:27.703 INFO:teuthology.orchestra.run.smithi158.stdout: re2-1:20211101-3.el9.x86_64 2023-10-12T02:10:27.703 INFO:teuthology.orchestra.run.smithi158.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-12T02:10:27.703 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:27.703 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:10:27.892 INFO:teuthology.orchestra.run.smithi125.stdout:(6/10): libcephsqlite-18.0.0-6684.g9587ffdf.el9 275 kB/s | 161 kB 00:00 2023-10-12T02:10:28.025 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-test 2023-10-12T02:10:28.067 INFO:teuthology.orchestra.run.smithi125.stdout:(7/10): cryptopp-8.6.0-1.el9.x86_64.rpm 7.2 MB/s | 1.3 MB 00:00 2023-10-12T02:10:28.109 INFO:teuthology.orchestra.run.smithi125.stdout:(8/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.8 MB/s | 122 kB 00:00 2023-10-12T02:10:28.209 INFO:teuthology.orchestra.run.smithi125.stdout:(9/10): jq-1.6-15.el9.x86_64.rpm 315 kB/s | 187 kB 00:00 2023-10-12T02:10:28.603 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:00:28 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:10:28.741 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:10:28.742 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:10:28.742 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:10:28.743 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:10:28.743 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:10:28.743 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 117 M 2023-10-12T02:10:28.743 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:10:28.743 INFO:teuthology.orchestra.run.smithi158.stdout: cryptopp x86_64 8.6.0-1.el9 epel 1.3 M 2023-10-12T02:10:28.744 INFO:teuthology.orchestra.run.smithi158.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-12T02:10:28.744 INFO:teuthology.orchestra.run.smithi158.stdout: libasan x86_64 11.4.1-2.3.el9 CentOS-AppStream 418 k 2023-10-12T02:10:28.744 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 161 k 2023-10-12T02:10:28.744 INFO:teuthology.orchestra.run.smithi158.stdout: libubsan x86_64 11.4.1-2.3.el9 CentOS-AppStream 186 k 2023-10-12T02:10:28.744 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-12T02:10:28.745 INFO:teuthology.orchestra.run.smithi158.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-12T02:10:28.745 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-12T02:10:28.745 INFO:teuthology.orchestra.run.smithi158.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2023-10-12T02:10:28.745 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:28.745 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:10:28.746 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:10:28.746 INFO:teuthology.orchestra.run.smithi158.stdout:Install 10 Packages 2023-10-12T02:10:28.746 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:10:28.747 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 120 M 2023-10-12T02:10:28.747 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 1.1 G 2023-10-12T02:10:28.747 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:10:29.363 INFO:teuthology.orchestra.run.smithi158.stdout:(1/10): libasan-11.4.1-2.3.el9.x86_64.rpm 3.1 MB/s | 418 kB 00:00 2023-10-12T02:10:29.430 INFO:teuthology.orchestra.run.smithi158.stdout:(2/10): libubsan-11.4.1-2.3.el9.x86_64.rpm 4.2 MB/s | 186 kB 00:00 2023-10-12T02:10:29.472 INFO:teuthology.orchestra.run.smithi158.stdout:(3/10): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2023-10-12T02:10:29.514 INFO:teuthology.orchestra.run.smithi158.stdout:(4/10): socat-1.7.4.1-5.el9.x86_64.rpm 7.2 MB/s | 300 kB 00:00 2023-10-12T02:10:29.548 INFO:teuthology.orchestra.run.smithi158.stdout:(5/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2023-10-12T02:10:29.690 INFO:teuthology.orchestra.run.smithi158.stdout:(6/10): jq-1.6-15.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-10-12T02:10:29.840 INFO:teuthology.orchestra.run.smithi158.stdout:(7/10): libcephsqlite-18.0.0-6684.g9587ffdf.el9 265 kB/s | 161 kB 00:00 2023-10-12T02:10:30.232 INFO:teuthology.orchestra.run.smithi158.stdout:(8/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 311 kB/s | 122 kB 00:00 2023-10-12T02:10:30.332 INFO:teuthology.orchestra.run.smithi158.stdout:(9/10): cryptopp-8.6.0-1.el9.x86_64.rpm 2.0 MB/s | 1.3 MB 00:00 2023-10-12T02:10:40.067 INFO:teuthology.orchestra.run.smithi125.stdout:(10/10): ceph-test-18.0.0-6684.g9587ffdf.el9.x8 9.2 MB/s | 117 MB 00:12 2023-10-12T02:10:40.069 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:10:40.069 INFO:teuthology.orchestra.run.smithi125.stdout:Total 9.3 MB/s | 120 MB 00:12 2023-10-12T02:10:40.083 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:10:40.104 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:10:40.105 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:10:40.915 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:10:40.916 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:10:42.012 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:10:42.105 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2023-10-12T02:10:42.141 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cryptopp-8.6.0-1.el9.x86_64 2/10 2023-10-12T02:10:42.184 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/10 2023-10-12T02:10:42.232 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/10 2023-10-12T02:10:42.267 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 5/10 2023-10-12T02:10:42.299 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : jq-1.6-15.el9.x86_64 6/10 2023-10-12T02:10:42.336 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libubsan-11.4.1-2.3.el9.x86_64 7/10 2023-10-12T02:10:42.378 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libasan-11.4.1-2.3.el9.x86_64 8/10 2023-10-12T02:10:42.404 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 9/10 2023-10-12T02:10:46.397 INFO:teuthology.orchestra.run.smithi158.stdout:(10/10): ceph-test-18.0.0-6684.g9587ffdf.el9.x8 6.8 MB/s | 117 MB 00:17 2023-10-12T02:10:46.400 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:10:46.400 INFO:teuthology.orchestra.run.smithi158.stdout:Total 6.8 MB/s | 120 MB 00:17 2023-10-12T02:10:46.414 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:10:46.437 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:10:46.437 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:10:47.260 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:10:47.261 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:10:48.388 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:10:48.489 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2023-10-12T02:10:48.530 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cryptopp-8.6.0-1.el9.x86_64 2/10 2023-10-12T02:10:48.573 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/10 2023-10-12T02:10:48.613 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/10 2023-10-12T02:10:48.648 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 5/10 2023-10-12T02:10:48.680 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : jq-1.6-15.el9.x86_64 6/10 2023-10-12T02:10:48.712 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libubsan-11.4.1-2.3.el9.x86_64 7/10 2023-10-12T02:10:48.760 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libasan-11.4.1-2.3.el9.x86_64 8/10 2023-10-12T02:10:48.786 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 9/10 2023-10-12T02:10:58.606 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 9/10 2023-10-12T02:10:58.685 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 10/10 2023-10-12T02:11:04.879 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 9/10 2023-10-12T02:11:04.949 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 10/10 2023-10-12T02:11:06.751 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 10/10 2023-10-12T02:11:06.751 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/10 2023-10-12T02:11:06.751 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/10 2023-10-12T02:11:06.751 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libasan-11.4.1-2.3.el9.x86_64 3/10 2023-10-12T02:11:06.752 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libubsan-11.4.1-2.3.el9.x86_64 4/10 2023-10-12T02:11:06.752 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 5/10 2023-10-12T02:11:06.752 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 6/10 2023-10-12T02:11:06.752 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 7/10 2023-10-12T02:11:06.752 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : jq-1.6-15.el9.x86_64 8/10 2023-10-12T02:11:06.753 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cryptopp-8.6.0-1.el9.x86_64 9/10 2023-10-12T02:11:07.198 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2023-10-12T02:11:07.199 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:07.199 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:11:07.199 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:07.199 INFO:teuthology.orchestra.run.smithi125.stdout: cryptopp-8.6.0-1.el9.x86_64 2023-10-12T02:11:07.199 INFO:teuthology.orchestra.run.smithi125.stdout: jq-1.6-15.el9.x86_64 2023-10-12T02:11:07.200 INFO:teuthology.orchestra.run.smithi125.stdout: libasan-11.4.1-2.3.el9.x86_64 2023-10-12T02:11:07.200 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:07.200 INFO:teuthology.orchestra.run.smithi125.stdout: libubsan-11.4.1-2.3.el9.x86_64 2023-10-12T02:11:07.200 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-12T02:11:07.200 INFO:teuthology.orchestra.run.smithi125.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-12T02:11:07.201 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-12T02:11:07.201 INFO:teuthology.orchestra.run.smithi125.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2023-10-12T02:11:07.201 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:07.201 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:11:07.487 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph 2023-10-12T02:11:08.063 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:11 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:11:08.204 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:11:08.206 INFO:teuthology.orchestra.run.smithi125.stdout:========================================================================================= 2023-10-12T02:11:08.206 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:11:08.206 INFO:teuthology.orchestra.run.smithi125.stdout:========================================================================================= 2023-10-12T02:11:08.207 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:11:08.207 INFO:teuthology.orchestra.run.smithi125.stdout: ceph x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 6.5 k 2023-10-12T02:11:08.207 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:11:08.207 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 2.0 M 2023-10-12T02:11:08.207 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 1.5 M 2023-10-12T02:11:08.208 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 242 k 2023-10-12T02:11:08.208 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 8.8 M 2023-10-12T02:11:08.208 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 117 M 2023-10-12T02:11:08.208 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-12T02:11:08.208 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-12T02:11:08.209 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-12T02:11:08.209 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-12T02:11:08.209 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-12T02:11:08.209 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-12T02:11:08.209 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-12T02:11:08.210 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-12T02:11:08.210 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-12T02:11:08.210 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-12T02:11:08.210 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-12T02:11:08.210 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-12T02:11:08.211 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-12T02:11:08.211 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-12T02:11:08.211 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-12T02:11:08.211 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-12T02:11:08.211 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-12T02:11:08.212 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-12T02:11:08.212 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-12T02:11:08.212 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-12T02:11:08.212 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-12T02:11:08.213 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-12T02:11:08.213 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-12T02:11:08.213 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-12T02:11:08.213 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-12T02:11:08.213 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-12T02:11:08.214 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-12T02:11:08.214 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:08.214 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:11:08.214 INFO:teuthology.orchestra.run.smithi125.stdout:========================================================================================= 2023-10-12T02:11:08.214 INFO:teuthology.orchestra.run.smithi125.stdout:Install 33 Packages 2023-10-12T02:11:08.215 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:08.215 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 134 M 2023-10-12T02:11:08.215 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 1.4 G 2023-10-12T02:11:08.216 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:11:09.118 INFO:teuthology.orchestra.run.smithi125.stdout:(1/33): ceph-18.0.0-6684.g9587ffdf.el9.x86_64.r 21 kB/s | 6.5 kB 00:00 2023-10-12T02:11:09.777 INFO:teuthology.orchestra.run.smithi125.stdout:(2/33): ceph-mgr-18.0.0-6684.g9587ffdf.el9.x86_ 1.5 MB/s | 1.5 MB 00:00 2023-10-12T02:11:10.027 INFO:teuthology.orchestra.run.smithi125.stdout:(3/33): ceph-mds-18.0.0-6684.g9587ffdf.el9.x86_ 1.6 MB/s | 2.0 MB 00:01 2023-10-12T02:11:10.210 INFO:teuthology.orchestra.run.smithi125.stdout:(4/33): ceph-mgr-modules-core-18.0.0-6684.g9587 1.3 MB/s | 242 kB 00:00 2023-10-12T02:11:10.435 INFO:teuthology.orchestra.run.smithi125.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 641 kB/s | 144 kB 00:00 2023-10-12T02:11:10.485 INFO:teuthology.orchestra.run.smithi125.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 633 kB/s | 32 kB 00:00 2023-10-12T02:11:10.536 INFO:teuthology.orchestra.run.smithi125.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 771 kB/s | 39 kB 00:00 2023-10-12T02:11:10.595 INFO:teuthology.orchestra.run.smithi125.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2023-10-12T02:11:10.645 INFO:teuthology.orchestra.run.smithi125.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2023-10-12T02:11:10.695 INFO:teuthology.orchestra.run.smithi125.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2023-10-12T02:11:10.745 INFO:teuthology.orchestra.run.smithi125.stdout:(11/33): python3-requests-2.25.1-7.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2023-10-12T02:11:10.804 INFO:teuthology.orchestra.run.smithi125.stdout:(12/33): python3-urllib3-1.26.5-3.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2023-10-12T02:11:11.004 INFO:teuthology.orchestra.run.smithi125.stdout:(13/33): python3-cryptography-36.0.1-4.el9.x86_ 6.1 MB/s | 1.2 MB 00:00 2023-10-12T02:11:11.180 INFO:teuthology.orchestra.run.smithi125.stdout:(14/33): ceph-mon-18.0.0-6684.g9587ffdf.el9.x86 4.3 MB/s | 8.8 MB 00:02 2023-10-12T02:11:11.538 INFO:teuthology.orchestra.run.smithi125.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 96 kB/s | 51 kB 00:00 2023-10-12T02:11:11.613 INFO:teuthology.orchestra.run.smithi125.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 106 kB/s | 43 kB 00:00 2023-10-12T02:11:12.013 INFO:teuthology.orchestra.run.smithi125.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 950 kB/s | 358 kB 00:00 2023-10-12T02:11:12.097 INFO:teuthology.orchestra.run.smithi125.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 307 kB/s | 172 kB 00:00 2023-10-12T02:11:12.124 INFO:teuthology.orchestra.run.smithi125.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 96 kB/s | 11 kB 00:00 2023-10-12T02:11:12.191 INFO:teuthology.orchestra.run.smithi125.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 188 kB/s | 18 kB 00:00 2023-10-12T02:11:12.225 INFO:teuthology.orchestra.run.smithi125.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2023-10-12T02:11:12.283 INFO:teuthology.orchestra.run.smithi125.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2023-10-12T02:11:12.325 INFO:teuthology.orchestra.run.smithi125.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2023-10-12T02:11:12.384 INFO:teuthology.orchestra.run.smithi125.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 462 kB/s | 46 kB 00:00 2023-10-12T02:11:12.434 INFO:teuthology.orchestra.run.smithi125.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 725 kB/s | 79 kB 00:00 2023-10-12T02:11:12.520 INFO:teuthology.orchestra.run.smithi125.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2023-10-12T02:11:12.543 INFO:teuthology.orchestra.run.smithi125.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2023-10-12T02:11:12.618 INFO:teuthology.orchestra.run.smithi125.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 898 kB/s | 90 kB 00:00 2023-10-12T02:11:12.644 INFO:teuthology.orchestra.run.smithi125.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 356 kB/s | 36 kB 00:00 2023-10-12T02:11:12.744 INFO:teuthology.orchestra.run.smithi125.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2023-10-12T02:11:12.844 INFO:teuthology.orchestra.run.smithi125.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 200 kB/s | 20 kB 00:00 2023-10-12T02:11:12.869 INFO:teuthology.orchestra.run.smithi125.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.9 MB/s | 427 kB 00:00 2023-10-12T02:11:13.999 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 10/10 2023-10-12T02:11:13.999 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/10 2023-10-12T02:11:13.999 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/10 2023-10-12T02:11:13.999 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libasan-11.4.1-2.3.el9.x86_64 3/10 2023-10-12T02:11:14.000 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libubsan-11.4.1-2.3.el9.x86_64 4/10 2023-10-12T02:11:14.000 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 5/10 2023-10-12T02:11:14.000 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 6/10 2023-10-12T02:11:14.000 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 7/10 2023-10-12T02:11:14.000 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : jq-1.6-15.el9.x86_64 8/10 2023-10-12T02:11:14.001 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cryptopp-8.6.0-1.el9.x86_64 9/10 2023-10-12T02:11:14.486 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2023-10-12T02:11:14.559 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:14.560 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:11:14.560 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:14.560 INFO:teuthology.orchestra.run.smithi158.stdout: cryptopp-8.6.0-1.el9.x86_64 2023-10-12T02:11:14.561 INFO:teuthology.orchestra.run.smithi158.stdout: jq-1.6-15.el9.x86_64 2023-10-12T02:11:14.561 INFO:teuthology.orchestra.run.smithi158.stdout: libasan-11.4.1-2.3.el9.x86_64 2023-10-12T02:11:14.561 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:14.561 INFO:teuthology.orchestra.run.smithi158.stdout: libubsan-11.4.1-2.3.el9.x86_64 2023-10-12T02:11:14.562 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-12T02:11:14.562 INFO:teuthology.orchestra.run.smithi158.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-12T02:11:14.562 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-12T02:11:14.562 INFO:teuthology.orchestra.run.smithi158.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2023-10-12T02:11:14.563 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:14.563 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:11:14.731 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph 2023-10-12T02:11:15.303 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:01:15 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:11:15.443 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:11:15.445 INFO:teuthology.orchestra.run.smithi158.stdout:========================================================================================= 2023-10-12T02:11:15.445 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:11:15.446 INFO:teuthology.orchestra.run.smithi158.stdout:========================================================================================= 2023-10-12T02:11:15.446 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:11:15.446 INFO:teuthology.orchestra.run.smithi158.stdout: ceph x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 6.5 k 2023-10-12T02:11:15.446 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:11:15.446 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 2.0 M 2023-10-12T02:11:15.447 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 1.5 M 2023-10-12T02:11:15.447 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 242 k 2023-10-12T02:11:15.447 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 8.8 M 2023-10-12T02:11:15.447 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 117 M 2023-10-12T02:11:15.447 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-12T02:11:15.448 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-12T02:11:15.448 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-12T02:11:15.448 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-12T02:11:15.448 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-12T02:11:15.448 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-12T02:11:15.449 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-12T02:11:15.449 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-12T02:11:15.449 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-12T02:11:15.449 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-12T02:11:15.449 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-12T02:11:15.450 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-12T02:11:15.450 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-12T02:11:15.450 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-12T02:11:15.450 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-12T02:11:15.450 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-12T02:11:15.451 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-12T02:11:15.451 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-12T02:11:15.451 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-12T02:11:15.451 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-12T02:11:15.451 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-12T02:11:15.452 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-12T02:11:15.452 INFO:teuthology.orchestra.run.smithi158.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-12T02:11:15.452 INFO:teuthology.orchestra.run.smithi158.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-12T02:11:15.452 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-12T02:11:15.453 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-12T02:11:15.453 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-12T02:11:15.453 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:15.453 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:11:15.453 INFO:teuthology.orchestra.run.smithi158.stdout:========================================================================================= 2023-10-12T02:11:15.453 INFO:teuthology.orchestra.run.smithi158.stdout:Install 33 Packages 2023-10-12T02:11:15.454 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:15.454 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 134 M 2023-10-12T02:11:15.455 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 1.4 G 2023-10-12T02:11:15.455 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:11:16.176 INFO:teuthology.orchestra.run.smithi158.stdout:(1/33): ceph-18.0.0-6684.g9587ffdf.el9.x86_64.r 12 kB/s | 6.5 kB 00:00 2023-10-12T02:11:16.276 INFO:teuthology.orchestra.run.smithi158.stdout:(2/33): ceph-mgr-18.0.0-6684.g9587ffdf.el9.x86_ 2.2 MB/s | 1.5 MB 00:00 2023-10-12T02:11:16.318 INFO:teuthology.orchestra.run.smithi158.stdout:(3/33): ceph-mds-18.0.0-6684.g9587ffdf.el9.x86_ 2.9 MB/s | 2.0 MB 00:00 2023-10-12T02:11:16.401 INFO:teuthology.orchestra.run.smithi158.stdout:(4/33): ceph-mgr-modules-core-18.0.0-6684.g9587 2.8 MB/s | 242 kB 00:00 2023-10-12T02:11:16.510 INFO:teuthology.orchestra.run.smithi158.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2023-10-12T02:11:16.543 INFO:teuthology.orchestra.run.smithi158.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 950 kB/s | 32 kB 00:00 2023-10-12T02:11:16.577 INFO:teuthology.orchestra.run.smithi158.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2023-10-12T02:11:16.627 INFO:teuthology.orchestra.run.smithi158.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-10-12T02:11:16.661 INFO:teuthology.orchestra.run.smithi158.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2023-10-12T02:11:16.703 INFO:teuthology.orchestra.run.smithi158.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2023-10-12T02:11:16.737 INFO:teuthology.orchestra.run.smithi158.stdout:(11/33): python3-requests-2.25.1-7.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2023-10-12T02:11:16.787 INFO:teuthology.orchestra.run.smithi158.stdout:(12/33): python3-urllib3-1.26.5-3.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2023-10-12T02:11:17.102 INFO:teuthology.orchestra.run.smithi125.stdout:(33/33): ceph-osd-18.0.0-6684.g9587ffdf.el9.x86 16 MB/s | 117 MB 00:07 2023-10-12T02:11:17.109 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:11:17.109 INFO:teuthology.orchestra.run.smithi125.stdout:Total 15 MB/s | 134 MB 00:08 2023-10-12T02:11:17.140 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:11:17.179 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:11:17.180 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:11:17.596 INFO:teuthology.orchestra.run.smithi158.stdout:(13/33): python3-cryptography-36.0.1-4.el9.x86_ 1.5 MB/s | 1.2 MB 00:00 2023-10-12T02:11:17.713 INFO:teuthology.orchestra.run.smithi158.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2023-10-12T02:11:18.149 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:11:18.150 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:11:18.205 INFO:teuthology.orchestra.run.smithi158.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 88 kB/s | 43 kB 00:00 2023-10-12T02:11:18.538 INFO:teuthology.orchestra.run.smithi158.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 516 kB/s | 172 kB 00:00 2023-10-12T02:11:18.813 INFO:teuthology.orchestra.run.smithi158.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2023-10-12T02:11:18.939 INFO:teuthology.orchestra.run.smithi158.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 85 kB/s | 11 kB 00:00 2023-10-12T02:11:19.064 INFO:teuthology.orchestra.run.smithi158.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2023-10-12T02:11:19.190 INFO:teuthology.orchestra.run.smithi158.stdout:(20/33): python3-jaraco-collections-3.0.0-8.el9 185 kB/s | 23 kB 00:00 2023-10-12T02:11:19.315 INFO:teuthology.orchestra.run.smithi158.stdout:(21/33): python3-jaraco-functools-3.5.0-2.el9.n 156 kB/s | 19 kB 00:00 2023-10-12T02:11:19.441 INFO:teuthology.orchestra.run.smithi158.stdout:(22/33): python3-jaraco-text-3.2.0-6.el9.noarch 157 kB/s | 20 kB 00:00 2023-10-12T02:11:19.565 INFO:teuthology.orchestra.run.smithi158.stdout:(23/33): python3-logutils-0.3.5-21.el9.noarch.r 370 kB/s | 46 kB 00:00 2023-10-12T02:11:19.699 INFO:teuthology.orchestra.run.smithi158.stdout:(24/33): python3-more-itertools-8.12.0-2.el9.no 592 kB/s | 79 kB 00:00 2023-10-12T02:11:19.858 INFO:teuthology.orchestra.run.smithi158.stdout:(25/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2023-10-12T02:11:19.953 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:11:19.983 INFO:teuthology.orchestra.run.smithi158.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 131 kB/s | 16 kB 00:00 2023-10-12T02:11:20.021 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-10-12T02:11:20.053 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-10-12T02:11:20.086 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-10-12T02:11:20.117 INFO:teuthology.orchestra.run.smithi158.stdout:(27/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 675 kB/s | 90 kB 00:00 2023-10-12T02:11:20.177 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-10-12T02:11:20.242 INFO:teuthology.orchestra.run.smithi158.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 286 kB/s | 36 kB 00:00 2023-10-12T02:11:20.264 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-10-12T02:11:20.313 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-10-12T02:11:20.360 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-10-12T02:11:20.400 INFO:teuthology.orchestra.run.smithi158.stdout:(29/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2023-10-12T02:11:20.436 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-10-12T02:11:20.490 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 9/33 2023-10-12T02:11:20.529 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-10-12T02:11:20.578 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-10-12T02:11:20.584 INFO:teuthology.orchestra.run.smithi158.stdout:(30/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.3 MB/s | 427 kB 00:00 2023-10-12T02:11:20.642 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-12T02:11:20.701 INFO:teuthology.orchestra.run.smithi158.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 171 kB/s | 20 kB 00:00 2023-10-12T02:11:20.824 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-10-12T02:11:20.867 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-10-12T02:11:20.897 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-10-12T02:11:20.932 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-10-12T02:11:20.962 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-10-12T02:11:21.037 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-10-12T02:11:21.072 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-10-12T02:11:21.105 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-10-12T02:11:21.147 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-10-12T02:11:21.173 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-10-12T02:11:21.252 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-10-12T02:11:21.312 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-10-12T02:11:21.383 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-10-12T02:11:21.520 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-10-12T02:11:21.637 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-12T02:11:21.806 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6684.g9587ffdf.el9. 28/33 2023-10-12T02:11:21.834 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/33 2023-10-12T02:11:21.859 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/33 2023-10-12T02:11:21.860 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:21.860 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-12T02:11:21.860 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-12T02:11:21.860 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-12T02:11:21.860 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:24.109 INFO:teuthology.orchestra.run.smithi158.stdout:(32/33): ceph-osd-18.0.0-6684.g9587ffdf.el9.x86 15 MB/s | 117 MB 00:07 2023-10-12T02:11:28.652 INFO:teuthology.orchestra.run.smithi158.stdout:(33/33): ceph-mon-18.0.0-6684.g9587ffdf.el9.x86 725 kB/s | 8.8 MB 00:12 2023-10-12T02:11:28.658 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:11:28.658 INFO:teuthology.orchestra.run.smithi158.stdout:Total 10 MB/s | 134 MB 00:13 2023-10-12T02:11:28.690 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:11:28.731 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:11:28.732 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:11:29.715 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:11:29.715 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:11:31.531 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:11:31.597 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-10-12T02:11:31.628 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-10-12T02:11:31.661 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-10-12T02:11:31.752 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-10-12T02:11:31.831 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-10-12T02:11:31.872 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-10-12T02:11:31.910 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-10-12T02:11:31.977 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-10-12T02:11:32.023 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 9/33 2023-10-12T02:11:32.062 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-10-12T02:11:32.118 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-10-12T02:11:32.174 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-12T02:11:32.360 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-10-12T02:11:32.416 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-10-12T02:11:32.446 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-10-12T02:11:32.481 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-10-12T02:11:32.511 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-10-12T02:11:32.587 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-10-12T02:11:32.622 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-10-12T02:11:32.655 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-10-12T02:11:32.696 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-10-12T02:11:32.723 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-10-12T02:11:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-10-12T02:11:32.862 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-10-12T02:11:32.933 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-10-12T02:11:33.071 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-10-12T02:11:33.188 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-12T02:11:33.364 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6684.g9587ffdf.el9. 28/33 2023-10-12T02:11:33.401 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/33 2023-10-12T02:11:33.426 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 29/33 2023-10-12T02:11:33.426 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:33.426 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-12T02:11:33.427 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-12T02:11:33.427 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-12T02:11:33.427 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:40.200 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/33 2023-10-12T02:11:40.226 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/33 2023-10-12T02:11:40.226 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:40.226 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-12T02:11:40.227 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-12T02:11:40.227 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-12T02:11:40.227 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:41.100 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 31/33 2023-10-12T02:11:41.123 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 31/33 2023-10-12T02:11:41.123 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:41.123 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-12T02:11:41.123 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-12T02:11:41.124 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-12T02:11:41.124 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:41.326 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 32/33 2023-10-12T02:11:41.348 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 32/33 2023-10-12T02:11:41.348 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:41.349 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-12T02:11:41.349 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-12T02:11:41.349 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-12T02:11:41.349 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:41.414 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 33/33 2023-10-12T02:11:51.020 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 33/33 2023-10-12T02:11:51.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/33 2023-10-12T02:11:51.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/33 2023-10-12T02:11:51.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 3/33 2023-10-12T02:11:51.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/33 2023-10-12T02:11:51.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 5/33 2023-10-12T02:11:51.022 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6684.g9587ffdf.el9. 6/33 2023-10-12T02:11:51.022 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-10-12T02:11:51.022 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-10-12T02:11:51.022 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-10-12T02:11:51.022 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-10-12T02:11:51.023 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-10-12T02:11:51.023 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-12T02:11:51.023 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-10-12T02:11:51.023 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 14/33 2023-10-12T02:11:51.024 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-10-12T02:11:51.024 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-10-12T02:11:51.024 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-10-12T02:11:51.025 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-10-12T02:11:51.025 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-10-12T02:11:51.025 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-10-12T02:11:51.025 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-10-12T02:11:51.026 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-10-12T02:11:51.026 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-10-12T02:11:51.026 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-10-12T02:11:51.026 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-10-12T02:11:51.026 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-10-12T02:11:51.027 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-12T02:11:51.027 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-10-12T02:11:51.027 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-10-12T02:11:51.027 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-10-12T02:11:51.027 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-10-12T02:11:51.028 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-10-12T02:11:51.659 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/33 2023-10-12T02:11:51.686 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 30/33 2023-10-12T02:11:51.686 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:51.686 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-12T02:11:51.687 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-12T02:11:51.687 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-12T02:11:51.687 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:52.108 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-10-12T02:11:52.108 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:52.108 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:11:52.109 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:52.109 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:52.109 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:52.110 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:11:52.110 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:52.111 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:52.111 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-12T02:11:52.111 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-12T02:11:52.111 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-12T02:11:52.112 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-12T02:11:52.112 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-12T02:11:52.112 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-12T02:11:52.113 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-12T02:11:52.113 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-12T02:11:52.113 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-12T02:11:52.113 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-12T02:11:52.113 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-12T02:11:52.114 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-12T02:11:52.114 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-12T02:11:52.114 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-12T02:11:52.115 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-12T02:11:52.115 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply-3.11-14.el9.noarch 2023-10-12T02:11:52.115 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-12T02:11:52.116 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-12T02:11:52.116 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-12T02:11:52.117 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-12T02:11:52.117 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-12T02:11:52.117 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-12T02:11:52.117 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-12T02:11:52.118 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-12T02:11:52.118 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-12T02:11:52.118 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-12T02:11:52.118 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-12T02:11:52.119 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:52.119 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:11:52.362 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-base 2023-10-12T02:11:52.617 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 31/33 2023-10-12T02:11:52.641 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 31/33 2023-10-12T02:11:52.642 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:52.642 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-12T02:11:52.642 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-12T02:11:52.643 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-12T02:11:52.643 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:52.852 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 32/33 2023-10-12T02:11:52.874 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 32/33 2023-10-12T02:11:52.875 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:52.875 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-12T02:11:52.875 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-12T02:11:52.875 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-12T02:11:52.875 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:11:52.943 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:55 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:11:52.948 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 33/33 2023-10-12T02:11:53.033 INFO:teuthology.orchestra.run.smithi125.stdout:Package ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:11:53.081 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:11:53.082 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:11:53.082 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:11:53.139 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install cephadm 2023-10-12T02:11:53.718 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:56 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:11:53.854 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:11:53.855 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:11:53.855 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:11:53.855 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:11:53.856 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:11:53.856 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 269 k 2023-10-12T02:11:53.856 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:53.856 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:11:53.856 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:11:53.857 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:11:53.857 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:53.857 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 269 k 2023-10-12T02:11:53.857 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 264 k 2023-10-12T02:11:53.857 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:11:54.506 INFO:teuthology.orchestra.run.smithi125.stdout:cephadm-18.0.0-6684.g9587ffdf.el9.noarch.rpm 413 kB/s | 269 kB 00:00 2023-10-12T02:11:54.507 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:11:54.507 INFO:teuthology.orchestra.run.smithi125.stdout:Total 412 kB/s | 269 kB 00:00 2023-10-12T02:11:54.508 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:11:54.512 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:11:54.512 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:11:54.515 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:11:54.516 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:11:54.590 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:11:54.986 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:11:55.067 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:11:55.470 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:11:55.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:11:55.763 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:55.764 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:11:55.764 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:11:55.764 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:55.764 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:11:55.977 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-immutable-object-cache 2023-10-12T02:11:56.560 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:59 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:11:56.701 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:11:56.702 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:11:56.703 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2023-10-12T02:11:56.703 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:11:56.703 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:11:56.703 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 144 k 2023-10-12T02:11:56.703 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:56.704 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:11:56.704 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:11:56.704 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:11:56.704 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:56.704 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 144 k 2023-10-12T02:11:56.705 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 438 k 2023-10-12T02:11:56.705 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:11:57.421 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-immutable-object-cache-18.0.0-6684.g9587ff 201 kB/s | 144 kB 00:00 2023-10-12T02:11:57.422 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:11:57.422 INFO:teuthology.orchestra.run.smithi125.stdout:Total 201 kB/s | 144 kB 00:00 2023-10-12T02:11:57.422 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:11:57.430 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:11:57.430 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:11:57.484 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:11:57.485 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:11:57.622 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:11:57.680 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf. 1/1 2023-10-12T02:11:57.704 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf. 1/1 2023-10-12T02:11:57.704 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:11:57.704 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-12T02:11:57.705 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:58.686 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf. 1/1 2023-10-12T02:11:58.687 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:58.687 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:11:58.687 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:11:58.687 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:11:58.688 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:11:58.942 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr 2023-10-12T02:11:59.533 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:02 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:11:59.622 INFO:teuthology.orchestra.run.smithi125.stdout:Package ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:11:59.670 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:11:59.671 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:11:59.671 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:11:59.731 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-dashboard 2023-10-12T02:12:00.302 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:03 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:00.440 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:00.440 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:00.441 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:12:00.441 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:00.441 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:00.441 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 1.8 M 2023-10-12T02:12:00.442 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:12:00.442 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 23 k 2023-10-12T02:12:00.442 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 15 k 2023-10-12T02:12:00.442 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-12T02:12:00.442 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-12T02:12:00.443 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-12T02:12:00.443 INFO:teuthology.orchestra.run.smithi125.stdout:Installing weak dependencies: 2023-10-12T02:12:00.443 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-12T02:12:00.443 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:00.443 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:00.444 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:00.444 INFO:teuthology.orchestra.run.smithi125.stdout:Install 7 Packages 2023-10-12T02:12:00.444 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 2.1 M 2023-10-12T02:12:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 8.6 M 2023-10-12T02:12:00.445 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:00.975 INFO:teuthology.orchestra.run.smithi125.stdout:(1/7): ceph-grafana-dashboards-18.0.0-6684.g958 63 kB/s | 23 kB 00:00 2023-10-12T02:12:01.017 INFO:teuthology.orchestra.run.smithi125.stdout:(2/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 217 kB/s | 9.0 kB 00:00 2023-10-12T02:12:01.042 INFO:teuthology.orchestra.run.smithi125.stdout:(3/7): ceph-prometheus-alerts-18.0.0-6684.g9587 34 kB/s | 15 kB 00:00 2023-10-12T02:12:01.076 INFO:teuthology.orchestra.run.smithi125.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 699 kB/s | 41 kB 00:00 2023-10-12T02:12:01.101 INFO:teuthology.orchestra.run.smithi125.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 527 kB/s | 31 kB 00:00 2023-10-12T02:12:01.135 INFO:teuthology.orchestra.run.smithi125.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2023-10-12T02:12:01.926 INFO:teuthology.orchestra.run.smithi125.stdout:(7/7): ceph-mgr-dashboard-18.0.0-6684.g9587ffdf 1.3 MB/s | 1.8 MB 00:01 2023-10-12T02:12:01.928 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:01.928 INFO:teuthology.orchestra.run.smithi125.stdout:Total 1.4 MB/s | 2.1 MB 00:01 2023-10-12T02:12:01.929 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:01.938 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:01.938 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:02.011 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:02.011 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:02.314 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:02.378 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-10-12T02:12:02.411 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-10-12T02:12:02.457 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-10-12T02:12:02.490 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-10-12T02:12:02.519 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6684.g9587ffdf.el9.n 5/7 2023-10-12T02:12:02.762 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6684.g9587ffdf.el9. 6/7 2023-10-12T02:12:02.797 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarc 7/7 2023-10-12T02:12:03.325 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarc 7/7 2023-10-12T02:12:03.325 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6684.g9587ffdf.el9. 1/7 2023-10-12T02:12:03.325 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarc 2/7 2023-10-12T02:12:03.325 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6684.g9587ffdf.el9.n 3/7 2023-10-12T02:12:03.326 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-10-12T02:12:03.326 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-10-12T02:12:03.326 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-10-12T02:12:03.430 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 33/33 2023-10-12T02:12:03.430 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/33 2023-10-12T02:12:03.430 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/33 2023-10-12T02:12:03.431 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 3/33 2023-10-12T02:12:03.431 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 4/33 2023-10-12T02:12:03.431 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 5/33 2023-10-12T02:12:03.431 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6684.g9587ffdf.el9. 6/33 2023-10-12T02:12:03.431 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-10-12T02:12:03.432 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-10-12T02:12:03.432 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-10-12T02:12:03.432 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-10-12T02:12:03.432 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-10-12T02:12:03.432 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-12T02:12:03.433 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-10-12T02:12:03.433 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 14/33 2023-10-12T02:12:03.433 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-10-12T02:12:03.433 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-10-12T02:12:03.433 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-10-12T02:12:03.434 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-10-12T02:12:03.434 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-10-12T02:12:03.434 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-10-12T02:12:03.434 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-10-12T02:12:03.434 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-10-12T02:12:03.435 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-10-12T02:12:03.435 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-10-12T02:12:03.435 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-10-12T02:12:03.435 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-10-12T02:12:03.435 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-12T02:12:03.436 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-10-12T02:12:03.436 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-10-12T02:12:03.436 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-10-12T02:12:03.436 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-10-12T02:12:03.436 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-10-12T02:12:03.635 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-10-12T02:12:03.635 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:03.636 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:03.636 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:03.636 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:03.636 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:03.637 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-12T02:12:03.637 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-12T02:12:03.637 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-12T02:12:03.637 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-12T02:12:03.637 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:03.638 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:03.859 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-12T02:12:04.291 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-10-12T02:12:04.291 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:04.291 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:04.292 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:04.292 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:04.292 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:04.292 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:04.292 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:04.293 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:04.293 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-12T02:12:04.293 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-12T02:12:04.293 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-12T02:12:04.293 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-12T02:12:04.294 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-12T02:12:04.294 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-12T02:12:04.294 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-12T02:12:04.294 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-12T02:12:04.295 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-12T02:12:04.295 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-12T02:12:04.295 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-12T02:12:04.295 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-12T02:12:04.295 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-12T02:12:04.296 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-12T02:12:04.296 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-12T02:12:04.296 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ply-3.11-14.el9.noarch 2023-10-12T02:12:04.296 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-12T02:12:04.296 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-12T02:12:04.296 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-12T02:12:04.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-12T02:12:04.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-12T02:12:04.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-12T02:12:04.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-12T02:12:04.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-12T02:12:04.298 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-12T02:12:04.298 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-12T02:12:04.298 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-12T02:12:04.298 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:04.298 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:04.437 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:07 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:04.541 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-base 2023-10-12T02:12:04.576 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:04.577 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2023-10-12T02:12:04.577 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:12:04.577 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2023-10-12T02:12:04.577 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:04.578 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 7.4 M 2023-10-12T02:12:04.578 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:12:04.578 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-12T02:12:04.578 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-12T02:12:04.579 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-12T02:12:04.579 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-12T02:12:04.579 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-12T02:12:04.579 INFO:teuthology.orchestra.run.smithi125.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-12T02:12:04.579 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-12T02:12:04.580 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-12T02:12:04.580 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-12T02:12:04.580 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-12T02:12:04.580 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-12T02:12:04.580 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:04.581 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:04.581 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2023-10-12T02:12:04.581 INFO:teuthology.orchestra.run.smithi125.stdout:Install 12 Packages 2023-10-12T02:12:04.581 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:04.582 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 37 M 2023-10-12T02:12:04.582 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 209 M 2023-10-12T02:12:04.582 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:04.683 INFO:teuthology.orchestra.run.smithi125.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 306 kB/s | 32 kB 00:00 2023-10-12T02:12:04.741 INFO:teuthology.orchestra.run.smithi125.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 493 kB/s | 17 kB 00:00 2023-10-12T02:12:04.775 INFO:teuthology.orchestra.run.smithi125.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2023-10-12T02:12:04.850 INFO:teuthology.orchestra.run.smithi125.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2023-10-12T02:12:04.934 INFO:teuthology.orchestra.run.smithi125.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.5 MB/s | 205 kB 00:00 2023-10-12T02:12:05.009 INFO:teuthology.orchestra.run.smithi125.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-10-12T02:12:05.093 INFO:teuthology.orchestra.run.smithi125.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-10-12T02:12:05.118 INFO:teuthology.orchestra.run.smithi125.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2023-10-12T02:12:05.119 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:05 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:05.207 INFO:teuthology.orchestra.run.smithi158.stdout:Package ceph-base-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:05.255 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:05.256 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:05.256 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:05.276 INFO:teuthology.orchestra.run.smithi125.stdout:(9/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2023-10-12T02:12:05.321 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install cephadm 2023-10-12T02:12:05.494 INFO:teuthology.orchestra.run.smithi125.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2023-10-12T02:12:05.518 INFO:teuthology.orchestra.run.smithi125.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 814 kB/s | 195 kB 00:00 2023-10-12T02:12:05.898 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:05 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:06.035 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:06.037 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:06.037 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:12:06.037 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:06.038 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:06.038 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 269 k 2023-10-12T02:12:06.038 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:06.038 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:06.039 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:06.039 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:12:06.039 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:06.040 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 269 k 2023-10-12T02:12:06.040 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 264 k 2023-10-12T02:12:06.040 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:06.817 INFO:teuthology.orchestra.run.smithi158.stdout:cephadm-18.0.0-6684.g9587ffdf.el9.noarch.rpm 345 kB/s | 269 kB 00:00 2023-10-12T02:12:06.818 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:06.819 INFO:teuthology.orchestra.run.smithi158.stdout:Total 344 kB/s | 269 kB 00:00 2023-10-12T02:12:06.819 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:06.824 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:06.825 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:06.828 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:06.828 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:06.905 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:07.369 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:07.465 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:07.843 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:08.134 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:08.135 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:08.135 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:08.135 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:08.136 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:08.136 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:08.357 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-immutable-object-cache 2023-10-12T02:12:08.668 INFO:teuthology.orchestra.run.smithi125.stdout:(12/12): ceph-mgr-diskprediction-local-18.0.0-6 1.8 MB/s | 7.4 MB 00:04 2023-10-12T02:12:08.671 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:08.672 INFO:teuthology.orchestra.run.smithi125.stdout:Total 9.1 MB/s | 37 MB 00:04 2023-10-12T02:12:08.672 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:08.696 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:08.696 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:08.944 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:08 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:09.085 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:09.086 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:09.086 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repo Size 2023-10-12T02:12:09.087 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:09.087 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:09.087 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 144 k 2023-10-12T02:12:09.087 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:09.087 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:09.088 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:09.088 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:12:09.088 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:09.088 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 144 k 2023-10-12T02:12:09.088 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 438 k 2023-10-12T02:12:09.089 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:09.124 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:09.125 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:09.698 INFO:teuthology.orchestra.run.smithi158.stdout:ceph-immutable-object-cache-18.0.0-6684.g9587ff 236 kB/s | 144 kB 00:00 2023-10-12T02:12:09.699 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:09.699 INFO:teuthology.orchestra.run.smithi158.stdout:Total 236 kB/s | 144 kB 00:00 2023-10-12T02:12:09.700 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:09.708 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:09.708 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:09.762 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:09.763 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:09.831 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:09.914 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:09.916 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-10-12T02:12:09.948 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-10-12T02:12:09.982 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf. 1/1 2023-10-12T02:12:10.007 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf. 1/1 2023-10-12T02:12:10.008 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:12:10.008 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-12T02:12:10.008 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:10.039 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-10-12T02:12:10.076 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-10-12T02:12:10.255 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-12T02:12:10.291 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-12T02:12:10.378 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-10-12T02:12:10.963 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf. 1/1 2023-10-12T02:12:10.963 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:10.963 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:10.964 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:10.964 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:10.964 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:11.009 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-10-12T02:12:11.120 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-10-12T02:12:11.241 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-mgr 2023-10-12T02:12:11.839 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:11 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:11.930 INFO:teuthology.orchestra.run.smithi158.stdout:Package ceph-mgr-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:11.978 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:11.979 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:11.979 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:12.045 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-mgr-dashboard 2023-10-12T02:12:12.347 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-10-12T02:12:12.621 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:12 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:12.759 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:12.760 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:12.760 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:12:12.761 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:12.761 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:12.761 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 1.8 M 2023-10-12T02:12:12.762 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:12:12.762 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 23 k 2023-10-12T02:12:12.762 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 15 k 2023-10-12T02:12:12.762 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-12T02:12:12.762 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-12T02:12:12.763 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-12T02:12:12.763 INFO:teuthology.orchestra.run.smithi158.stdout:Installing weak dependencies: 2023-10-12T02:12:12.763 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-12T02:12:12.763 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:12.764 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:12.764 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:12.764 INFO:teuthology.orchestra.run.smithi158.stdout:Install 7 Packages 2023-10-12T02:12:12.764 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:12.765 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 2.1 M 2023-10-12T02:12:12.765 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 8.6 M 2023-10-12T02:12:12.765 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:13.186 INFO:teuthology.orchestra.run.smithi158.stdout:(1/7): ceph-grafana-dashboards-18.0.0-6684.g958 103 kB/s | 23 kB 00:00 2023-10-12T02:12:13.244 INFO:teuthology.orchestra.run.smithi158.stdout:(2/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 155 kB/s | 9.0 kB 00:00 2023-10-12T02:12:13.303 INFO:teuthology.orchestra.run.smithi158.stdout:(3/7): python3-jwt-2.4.0-1.el9.noarch.rpm 700 kB/s | 41 kB 00:00 2023-10-12T02:12:13.328 INFO:teuthology.orchestra.run.smithi158.stdout:(4/7): ceph-prometheus-alerts-18.0.0-6684.g9587 40 kB/s | 15 kB 00:00 2023-10-12T02:12:13.354 INFO:teuthology.orchestra.run.smithi158.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 612 kB/s | 31 kB 00:00 2023-10-12T02:12:13.388 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-10-12T02:12:13.416 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587f 12/12 2023-10-12T02:12:13.462 INFO:teuthology.orchestra.run.smithi158.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.4 MB/s | 188 kB 00:00 2023-10-12T02:12:13.596 INFO:teuthology.orchestra.run.smithi158.stdout:(7/7): ceph-mgr-dashboard-18.0.0-6684.g9587ffdf 2.8 MB/s | 1.8 MB 00:00 2023-10-12T02:12:13.597 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:13.598 INFO:teuthology.orchestra.run.smithi158.stdout:Total 2.5 MB/s | 2.1 MB 00:00 2023-10-12T02:12:13.598 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:13.606 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:13.606 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:13.667 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:13.667 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:13.964 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:14.039 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-10-12T02:12:14.073 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-10-12T02:12:14.119 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-10-12T02:12:14.159 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-10-12T02:12:14.188 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6684.g9587ffdf.el9.n 5/7 2023-10-12T02:12:14.424 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6684.g9587ffdf.el9. 6/7 2023-10-12T02:12:14.458 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarc 7/7 2023-10-12T02:12:15.033 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarc 7/7 2023-10-12T02:12:15.034 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6684.g9587ffdf.el9. 1/7 2023-10-12T02:12:15.034 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarc 2/7 2023-10-12T02:12:15.034 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6684.g9587ffdf.el9.n 3/7 2023-10-12T02:12:15.035 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-10-12T02:12:15.035 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-10-12T02:12:15.035 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-10-12T02:12:15.339 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-10-12T02:12:15.339 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:15.340 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:15.340 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:15.340 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:15.340 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:15.340 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-12T02:12:15.341 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-12T02:12:15.341 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-12T02:12:15.341 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-12T02:12:15.341 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:15.342 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:15.577 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-12T02:12:15.873 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587f 12/12 2023-10-12T02:12:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587f 1/12 2023-10-12T02:12:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-10-12T02:12:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-10-12T02:12:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-10-12T02:12:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-12T02:12:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-12T02:12:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-10-12T02:12:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-10-12T02:12:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-10-12T02:12:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-10-12T02:12:15.877 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-10-12T02:12:16.154 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:16 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:16.280 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-10-12T02:12:16.280 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:16.280 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:16.281 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:16.281 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-12T02:12:16.281 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-12T02:12:16.281 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-12T02:12:16.281 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-12T02:12:16.282 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-12T02:12:16.282 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-12T02:12:16.282 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-12T02:12:16.282 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-12T02:12:16.282 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-12T02:12:16.283 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-12T02:12:16.283 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-12T02:12:16.283 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:16.283 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:16.293 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:16.294 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2023-10-12T02:12:16.294 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:12:16.295 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2023-10-12T02:12:16.295 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:16.295 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 7.4 M 2023-10-12T02:12:16.295 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:12:16.295 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-12T02:12:16.296 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-12T02:12:16.296 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-12T02:12:16.296 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-12T02:12:16.296 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-12T02:12:16.296 INFO:teuthology.orchestra.run.smithi158.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-12T02:12:16.297 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-12T02:12:16.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-12T02:12:16.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-12T02:12:16.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-12T02:12:16.297 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-12T02:12:16.298 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:16.298 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:16.298 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2023-10-12T02:12:16.298 INFO:teuthology.orchestra.run.smithi158.stdout:Install 12 Packages 2023-10-12T02:12:16.298 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:16.299 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 37 M 2023-10-12T02:12:16.299 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 209 M 2023-10-12T02:12:16.299 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:16.452 INFO:teuthology.orchestra.run.smithi158.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 203 kB/s | 32 kB 00:00 2023-10-12T02:12:16.494 INFO:teuthology.orchestra.run.smithi158.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 413 kB/s | 17 kB 00:00 2023-10-12T02:12:16.570 INFO:teuthology.orchestra.run.smithi158.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2023-10-12T02:12:16.595 INFO:teuthology.orchestra.run.smithi158.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 34 kB 00:00 2023-10-12T02:12:16.670 INFO:teuthology.orchestra.run.smithi158.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.7 MB/s | 205 kB 00:00 2023-10-12T02:12:16.707 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-rook 2023-10-12T02:12:16.762 INFO:teuthology.orchestra.run.smithi158.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2023-10-12T02:12:16.854 INFO:teuthology.orchestra.run.smithi158.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2023-10-12T02:12:16.879 INFO:teuthology.orchestra.run.smithi158.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.9 MB/s | 217 kB 00:00 2023-10-12T02:12:17.005 INFO:teuthology.orchestra.run.smithi158.stdout:(9/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 6.4 MB/s | 803 kB 00:00 2023-10-12T02:12:17.230 INFO:teuthology.orchestra.run.smithi158.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2023-10-12T02:12:17.264 INFO:teuthology.orchestra.run.smithi158.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 762 kB/s | 195 kB 00:00 2023-10-12T02:12:17.294 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:20 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:17.434 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:17.435 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2023-10-12T02:12:17.435 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:12:17.435 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2023-10-12T02:12:17.435 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:17.436 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 47 k 2023-10-12T02:12:17.436 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:12:17.436 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-12T02:12:17.436 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-12T02:12:17.436 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-12T02:12:17.437 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-12T02:12:17.437 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-12T02:12:17.437 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-12T02:12:17.437 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-12T02:12:17.438 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-12T02:12:17.438 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-12T02:12:17.438 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-12T02:12:17.438 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-12T02:12:17.438 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-12T02:12:17.439 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:17.439 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:17.439 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2023-10-12T02:12:17.439 INFO:teuthology.orchestra.run.smithi125.stdout:Install 13 Packages 2023-10-12T02:12:17.439 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 2.0 M 2023-10-12T02:12:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 26 M 2023-10-12T02:12:17.441 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:17.632 INFO:teuthology.orchestra.run.smithi125.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 210 kB/s | 24 kB 00:00 2023-10-12T02:12:17.657 INFO:teuthology.orchestra.run.smithi125.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 128 kB/s | 18 kB 00:00 2023-10-12T02:12:17.708 INFO:teuthology.orchestra.run.smithi125.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2023-10-12T02:12:17.733 INFO:teuthology.orchestra.run.smithi125.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-10-12T02:12:17.758 INFO:teuthology.orchestra.run.smithi125.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2023-10-12T02:12:17.783 INFO:teuthology.orchestra.run.smithi125.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 858 kB/s | 43 kB 00:00 2023-10-12T02:12:17.859 INFO:teuthology.orchestra.run.smithi125.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 322 kB/s | 32 kB 00:00 2023-10-12T02:12:17.884 INFO:teuthology.orchestra.run.smithi125.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 149 kB/s | 15 kB 00:00 2023-10-12T02:12:17.976 INFO:teuthology.orchestra.run.smithi125.stdout:(9/13): python3-google-auth-2.23.2-1.el9.noarch 1.6 MB/s | 196 kB 00:00 2023-10-12T02:12:18.001 INFO:teuthology.orchestra.run.smithi125.stdout:(10/13): ceph-mgr-rook-18.0.0-6684.g9587ffdf.el 98 kB/s | 47 kB 00:00 2023-10-12T02:12:18.076 INFO:teuthology.orchestra.run.smithi125.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2023-10-12T02:12:18.102 INFO:teuthology.orchestra.run.smithi125.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2023-10-12T02:12:18.127 INFO:teuthology.orchestra.run.smithi125.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 713 kB/s | 90 kB 00:00 2023-10-12T02:12:18.129 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:18.129 INFO:teuthology.orchestra.run.smithi125.stdout:Total 2.9 MB/s | 2.0 MB 00:00 2023-10-12T02:12:18.130 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:18.142 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:18.143 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:18.237 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:18.238 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:18.747 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:18.917 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-10-12T02:12:18.970 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-10-12T02:12:19.026 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-10-12T02:12:19.063 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-10-12T02:12:19.104 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-10-12T02:12:19.198 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-10-12T02:12:19.299 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 7/13 2023-10-12T02:12:19.355 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-10-12T02:12:19.756 INFO:teuthology.orchestra.run.smithi158.stdout:(12/12): ceph-mgr-diskprediction-local-18.0.0-6 2.1 MB/s | 7.4 MB 00:03 2023-10-12T02:12:19.759 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:19.760 INFO:teuthology.orchestra.run.smithi158.stdout:Total 11 MB/s | 37 MB 00:03 2023-10-12T02:12:19.760 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:19.784 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:19.785 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:20.083 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-10-12T02:12:20.144 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-10-12T02:12:20.178 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-10-12T02:12:20.220 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:20.220 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:20.223 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-10-12T02:12:20.248 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 13/13 2023-10-12T02:12:20.940 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:20.971 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 13/13 2023-10-12T02:12:20.971 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 1/13 2023-10-12T02:12:20.972 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-10-12T02:12:20.972 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-10-12T02:12:20.972 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-10-12T02:12:20.972 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-10-12T02:12:20.972 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-10-12T02:12:20.973 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-10-12T02:12:20.973 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-10-12T02:12:20.973 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-10-12T02:12:20.973 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 10/13 2023-10-12T02:12:20.973 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-10-12T02:12:20.974 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-10-12T02:12:21.020 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-10-12T02:12:21.051 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-10-12T02:12:21.145 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-10-12T02:12:21.188 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-10-12T02:12:21.359 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-10-12T02:12:21.359 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:21.359 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:21.360 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:21.360 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-12T02:12:21.360 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-12T02:12:21.360 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-12T02:12:21.361 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-12T02:12:21.361 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-12T02:12:21.361 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-12T02:12:21.361 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-12T02:12:21.361 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-12T02:12:21.362 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-12T02:12:21.362 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-12T02:12:21.362 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-12T02:12:21.362 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-12T02:12:21.362 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:21.363 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:21.377 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-12T02:12:21.413 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-12T02:12:21.518 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-10-12T02:12:21.613 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-cephadm 2023-10-12T02:12:22.183 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-10-12T02:12:22.191 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:25 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:22.302 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-10-12T02:12:22.330 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:22.331 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2023-10-12T02:12:22.331 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:12:22.331 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2023-10-12T02:12:22.331 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:22.331 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 132 k 2023-10-12T02:12:22.332 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:12:22.332 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-12T02:12:22.332 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-12T02:12:22.332 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-12T02:12:22.332 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-12T02:12:22.333 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-12T02:12:22.333 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:22.333 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:22.333 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2023-10-12T02:12:22.333 INFO:teuthology.orchestra.run.smithi125.stdout:Install 6 Packages 2023-10-12T02:12:22.334 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:22.334 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 6.8 M 2023-10-12T02:12:22.335 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 33 M 2023-10-12T02:12:22.335 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:22.553 INFO:teuthology.orchestra.run.smithi125.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.3 MB/s | 228 kB 00:00 2023-10-12T02:12:22.712 INFO:teuthology.orchestra.run.smithi125.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-10-12T02:12:22.945 INFO:teuthology.orchestra.run.smithi125.stdout:(3/6): ceph-mgr-cephadm-18.0.0-6684.g9587ffdf.e 234 kB/s | 132 kB 00:00 2023-10-12T02:12:23.021 INFO:teuthology.orchestra.run.smithi125.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 187 kB/s | 58 kB 00:00 2023-10-12T02:12:23.162 INFO:teuthology.orchestra.run.smithi125.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 902 kB/s | 548 kB 00:00 2023-10-12T02:12:23.263 INFO:teuthology.orchestra.run.smithi125.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 161 kB/s | 51 kB 00:00 2023-10-12T02:12:23.264 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:23.265 INFO:teuthology.orchestra.run.smithi125.stdout:Total 7.3 MB/s | 6.8 MB 00:00 2023-10-12T02:12:23.265 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:23.274 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:23.274 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:23.363 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:23.363 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:23.538 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-10-12T02:12:23.663 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:23.833 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-10-12T02:12:23.888 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-10-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-10-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-10-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-10-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-10-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 6/6 2023-10-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587f 12/12 2023-10-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 6/6 2023-10-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/6 2023-10-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-10-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-10-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-10-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-10-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-10-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:25.950 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-fuse 2023-10-12T02:12:26.522 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:29 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:26.659 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:26.660 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:26.660 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:12:26.661 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:26.661 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:26.661 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 832 k 2023-10-12T02:12:26.662 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-12T02:12:26.662 INFO:teuthology.orchestra.run.smithi125.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-12T02:12:26.662 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:26.663 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:26.663 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:26.663 INFO:teuthology.orchestra.run.smithi125.stdout:Install 2 Packages 2023-10-12T02:12:26.664 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:26.664 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 911 k 2023-10-12T02:12:26.664 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 3.0 M 2023-10-12T02:12:26.664 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:26.764 INFO:teuthology.orchestra.run.smithi125.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 762 kB/s | 79 kB 00:00 2023-10-12T02:12:27.376 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587f 12/12 2023-10-12T02:12:27.376 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587f 1/12 2023-10-12T02:12:27.376 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-10-12T02:12:27.377 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-10-12T02:12:27.377 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-10-12T02:12:27.377 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-12T02:12:27.377 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-12T02:12:27.377 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-10-12T02:12:27.378 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-10-12T02:12:27.378 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-10-12T02:12:27.378 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-10-12T02:12:27.378 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-10-12T02:12:27.835 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-10-12T02:12:27.835 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:27.836 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:27.836 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:27.836 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-12T02:12:27.836 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-12T02:12:27.837 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-12T02:12:27.837 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-12T02:12:27.837 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-12T02:12:27.837 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-12T02:12:27.837 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-12T02:12:27.838 INFO:teuthology.orchestra.run.smithi158.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-12T02:12:27.838 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-12T02:12:27.838 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-12T02:12:27.838 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-12T02:12:27.838 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:27.839 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:28.048 INFO:teuthology.orchestra.run.smithi125.stdout:(2/2): ceph-fuse-18.0.0-6684.g9587ffdf.el9.x86_ 600 kB/s | 832 kB 00:01 2023-10-12T02:12:28.049 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:28.049 INFO:teuthology.orchestra.run.smithi125.stdout:Total 656 kB/s | 911 kB 00:01 2023-10-12T02:12:28.050 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:28.073 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:28.073 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:28.151 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:28.152 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:28.206 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-mgr-rook 2023-10-12T02:12:28.339 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:28.464 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-10-12T02:12:28.530 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/2 2023-10-12T02:12:28.790 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:28 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:28.929 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:28.931 INFO:teuthology.orchestra.run.smithi158.stdout:======================================================================================== 2023-10-12T02:12:28.931 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:12:28.931 INFO:teuthology.orchestra.run.smithi158.stdout:======================================================================================== 2023-10-12T02:12:28.931 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:28.931 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 47 k 2023-10-12T02:12:28.932 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:12:28.932 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-12T02:12:28.932 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-12T02:12:28.932 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-12T02:12:28.932 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-12T02:12:28.933 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-12T02:12:28.933 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-12T02:12:28.933 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-12T02:12:28.933 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-12T02:12:28.934 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-12T02:12:28.934 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-12T02:12:28.934 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-12T02:12:28.934 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-12T02:12:28.934 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:28.935 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:28.935 INFO:teuthology.orchestra.run.smithi158.stdout:======================================================================================== 2023-10-12T02:12:28.935 INFO:teuthology.orchestra.run.smithi158.stdout:Install 13 Packages 2023-10-12T02:12:28.935 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:28.936 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 2.0 M 2023-10-12T02:12:28.936 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 26 M 2023-10-12T02:12:28.936 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:29.182 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/2 2023-10-12T02:12:29.182 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/2 2023-10-12T02:12:29.222 INFO:teuthology.orchestra.run.smithi158.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 250 kB/s | 24 kB 00:00 2023-10-12T02:12:29.264 INFO:teuthology.orchestra.run.smithi158.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 130 kB/s | 18 kB 00:00 2023-10-12T02:12:29.314 INFO:teuthology.orchestra.run.smithi158.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.5 MB/s | 174 kB 00:00 2023-10-12T02:12:29.348 INFO:teuthology.orchestra.run.smithi158.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2023-10-12T02:12:29.373 INFO:teuthology.orchestra.run.smithi158.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2023-10-12T02:12:29.390 INFO:teuthology.orchestra.run.smithi158.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 43 kB 00:00 2023-10-12T02:12:29.480 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-10-12T02:12:29.481 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:29.481 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:29.481 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-10-12T02:12:29.481 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:29.481 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:29.632 INFO:teuthology.orchestra.run.smithi158.stdout:(7/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2023-10-12T02:12:29.649 INFO:teuthology.orchestra.run.smithi158.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 117 kB/s | 32 kB 00:00 2023-10-12T02:12:29.666 INFO:teuthology.orchestra.run.smithi158.stdout:(9/13): ceph-mgr-rook-18.0.0-6684.g9587ffdf.el9 88 kB/s | 47 kB 00:00 2023-10-12T02:12:29.716 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-volume 2023-10-12T02:12:29.858 INFO:teuthology.orchestra.run.smithi158.stdout:(10/13): python3-google-auth-2.23.2-1.el9.noarc 872 kB/s | 196 kB 00:00 2023-10-12T02:12:29.950 INFO:teuthology.orchestra.run.smithi158.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 979 kB/s | 90 kB 00:00 2023-10-12T02:12:29.992 INFO:teuthology.orchestra.run.smithi158.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.0 MB/s | 1.0 MB 00:00 2023-10-12T02:12:30.009 INFO:teuthology.orchestra.run.smithi158.stdout:(13/13): python3-rsa-4.9-2.el9.noarch.rpm 172 kB/s | 59 kB 00:00 2023-10-12T02:12:30.013 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:30.013 INFO:teuthology.orchestra.run.smithi158.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2023-10-12T02:12:30.014 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:30.026 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:30.026 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:30.122 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:30.123 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:30.306 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:33 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:30.444 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:30.445 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:30.445 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-12T02:12:30.446 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:30.446 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:30.446 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 262 k 2023-10-12T02:12:30.446 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:30.446 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:30.447 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:30.447 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:12:30.447 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:30.447 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 262 k 2023-10-12T02:12:30.447 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 1.2 M 2023-10-12T02:12:30.448 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:30.660 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:30.832 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-10-12T02:12:30.893 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-10-12T02:12:30.902 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-volume-18.0.0-6684.g9587ffdf.el9.noarch.rp 575 kB/s | 262 kB 00:00 2023-10-12T02:12:30.903 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:30.903 INFO:teuthology.orchestra.run.smithi125.stdout:Total 573 kB/s | 262 kB 00:00 2023-10-12T02:12:30.903 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:30.911 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:30.911 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:30.947 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:30.948 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:30.949 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-10-12T02:12:30.986 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-10-12T02:12:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-10-12T02:12:31.120 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-10-12T02:12:31.159 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:31.212 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:31.221 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 7/13 2023-10-12T02:12:31.227 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:31.228 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:12:31.228 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-12T02:12:31.228 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:31.278 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-10-12T02:12:31.994 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-10-12T02:12:32.052 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-10-12T02:12:32.086 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-10-12T02:12:32.131 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-10-12T02:12:32.155 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 13/13 2023-10-12T02:12:32.302 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:32.302 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:32.302 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:32.302 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:32.303 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:32.303 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:32.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install librados-devel 2023-10-12T02:12:32.968 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 13/13 2023-10-12T02:12:32.968 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 1/13 2023-10-12T02:12:32.968 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-10-12T02:12:32.969 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-10-12T02:12:32.969 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-10-12T02:12:32.969 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-10-12T02:12:32.969 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-10-12T02:12:32.969 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-10-12T02:12:32.970 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-10-12T02:12:32.970 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-10-12T02:12:32.970 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 10/13 2023-10-12T02:12:32.970 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-10-12T02:12:32.970 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-10-12T02:12:33.228 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:36 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:33.347 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-10-12T02:12:33.347 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:33.347 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:33.348 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:33.348 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-12T02:12:33.348 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-12T02:12:33.348 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-12T02:12:33.348 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-12T02:12:33.349 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-12T02:12:33.349 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-12T02:12:33.349 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-12T02:12:33.349 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-12T02:12:33.349 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-12T02:12:33.350 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-12T02:12:33.350 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-12T02:12:33.350 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-12T02:12:33.350 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:33.350 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:33.366 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:33.367 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:33.368 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2023-10-12T02:12:33.368 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:33.368 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:33.368 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 127 k 2023-10-12T02:12:33.368 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:33.369 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:33.369 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:33.369 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:12:33.369 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:33.369 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 127 k 2023-10-12T02:12:33.370 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 460 k 2023-10-12T02:12:33.370 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:33.610 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-mgr-cephadm 2023-10-12T02:12:33.832 INFO:teuthology.orchestra.run.smithi125.stdout:librados-devel-18.0.0-6684.g9587ffdf.el9.x86_64 274 kB/s | 127 kB 00:00 2023-10-12T02:12:33.832 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:33.833 INFO:teuthology.orchestra.run.smithi125.stdout:Total 273 kB/s | 127 kB 00:00 2023-10-12T02:12:33.833 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:33.838 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:33.838 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:33.898 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:33.899 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:34.047 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:34.147 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:34.188 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:34 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:34.328 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:34.328 INFO:teuthology.orchestra.run.smithi158.stdout:======================================================================================== 2023-10-12T02:12:34.328 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:12:34.329 INFO:teuthology.orchestra.run.smithi158.stdout:======================================================================================== 2023-10-12T02:12:34.329 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:34.329 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 132 k 2023-10-12T02:12:34.329 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:12:34.329 INFO:teuthology.orchestra.run.smithi158.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-12T02:12:34.330 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-12T02:12:34.330 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-12T02:12:34.330 INFO:teuthology.orchestra.run.smithi158.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-12T02:12:34.330 INFO:teuthology.orchestra.run.smithi158.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-12T02:12:34.331 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:34.331 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:34.331 INFO:teuthology.orchestra.run.smithi158.stdout:======================================================================================== 2023-10-12T02:12:34.331 INFO:teuthology.orchestra.run.smithi158.stdout:Install 6 Packages 2023-10-12T02:12:34.333 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:34.333 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 6.8 M 2023-10-12T02:12:34.334 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 33 M 2023-10-12T02:12:34.334 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:34.606 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:34.653 INFO:teuthology.orchestra.run.smithi158.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 858 kB/s | 228 kB 00:00 2023-10-12T02:12:34.876 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:34.876 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:34.877 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:34.877 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:34.877 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:34.877 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:34.878 INFO:teuthology.orchestra.run.smithi158.stdout:(2/6): ceph-mgr-cephadm-18.0.0-6684.g9587ffdf.e 269 kB/s | 132 kB 00:00 2023-10-12T02:12:34.995 INFO:teuthology.orchestra.run.smithi158.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 9.5 MB/s | 5.8 MB 00:00 2023-10-12T02:12:35.097 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install libcephfs2 2023-10-12T02:12:35.279 INFO:teuthology.orchestra.run.smithi158.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 144 kB/s | 58 kB 00:00 2023-10-12T02:12:35.337 INFO:teuthology.orchestra.run.smithi158.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 149 kB/s | 51 kB 00:00 2023-10-12T02:12:35.690 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:38 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:35.783 INFO:teuthology.orchestra.run.smithi125.stdout:Package libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:35.835 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:35.836 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:12:35.836 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:35.900 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install libcephfs-devel 2023-10-12T02:12:36.472 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:39 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:36.611 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:36.612 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:36.613 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2023-10-12T02:12:36.613 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:36.613 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:36.613 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 31 k 2023-10-12T02:12:36.613 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:36.614 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:36.614 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:36.614 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:12:36.614 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:36.614 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 31 k 2023-10-12T02:12:36.615 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 139 k 2023-10-12T02:12:36.615 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:36.860 INFO:teuthology.orchestra.run.smithi125.stdout:libcephfs-devel-18.0.0-6684.g9587ffdf.el9.x86_6 126 kB/s | 31 kB 00:00 2023-10-12T02:12:36.861 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:36.861 INFO:teuthology.orchestra.run.smithi125.stdout:Total 125 kB/s | 31 kB 00:00 2023-10-12T02:12:36.861 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:36.864 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:36.864 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:36.895 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:36.895 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:36.981 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:37.099 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:37.484 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:37.746 INFO:teuthology.orchestra.run.smithi158.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 177 kB/s | 548 kB 00:03 2023-10-12T02:12:37.748 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:37.749 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:37.749 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:37.749 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:37.749 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:37.750 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:37.750 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:37.750 INFO:teuthology.orchestra.run.smithi158.stdout:Total 2.0 MB/s | 6.8 MB 00:03 2023-10-12T02:12:37.750 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:37.759 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:37.759 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:37.851 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:37.851 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:37.937 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install librados2 2023-10-12T02:12:38.152 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-10-12T02:12:38.363 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-10-12T02:12:38.518 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:41 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:38.607 INFO:teuthology.orchestra.run.smithi125.stdout:Package librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:38.655 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:38.656 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:12:38.656 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:38.724 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install librbd1 2023-10-12T02:12:38.871 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-10-12T02:12:38.965 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-10-12T02:12:39.035 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-10-12T02:12:39.068 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 6/6 2023-10-12T02:12:39.297 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:42 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:39.385 INFO:teuthology.orchestra.run.smithi125.stdout:Package librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:39.434 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:39.435 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:12:39.435 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:39.495 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-rados 2023-10-12T02:12:39.908 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 6/6 2023-10-12T02:12:39.908 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 1/6 2023-10-12T02:12:39.909 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-10-12T02:12:39.909 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-10-12T02:12:39.909 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-10-12T02:12:39.909 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-10-12T02:12:40.076 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:43 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:40.167 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:40.216 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:40.217 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:12:40.217 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:40.219 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-10-12T02:12:40.219 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:40.219 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:40.220 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:40.220 INFO:teuthology.orchestra.run.smithi158.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-12T02:12:40.220 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-12T02:12:40.220 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-12T02:12:40.220 INFO:teuthology.orchestra.run.smithi158.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-12T02:12:40.221 INFO:teuthology.orchestra.run.smithi158.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-12T02:12:40.221 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:40.221 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:40.291 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-rgw 2023-10-12T02:12:40.505 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-fuse 2023-10-12T02:12:40.869 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:43 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:40.958 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:41.009 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:41.010 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:12:41.010 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:41.073 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-cephfs 2023-10-12T02:12:41.087 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:41 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:41.226 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:41.227 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:41.227 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:12:41.227 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:41.227 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:41.228 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 832 k 2023-10-12T02:12:41.228 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2023-10-12T02:12:41.228 INFO:teuthology.orchestra.run.smithi158.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-12T02:12:41.228 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:41.228 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:41.229 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:41.229 INFO:teuthology.orchestra.run.smithi158.stdout:Install 2 Packages 2023-10-12T02:12:41.229 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:41.229 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 911 k 2023-10-12T02:12:41.229 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 3.0 M 2023-10-12T02:12:41.230 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:41.347 INFO:teuthology.orchestra.run.smithi158.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 658 kB/s | 79 kB 00:00 2023-10-12T02:12:41.648 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:44 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi158.stdout:(2/2): ceph-fuse-18.0.0-6684.g9587ffdf.el9.x86_ 1.7 MB/s | 832 kB 00:00 2023-10-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi158.stdout:Total 1.9 MB/s | 911 kB 00:00 2023-10-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:41.738 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:41.788 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:41.789 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:12:41.789 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:41.804 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:41.804 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:41.866 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-rbd 2023-10-12T02:12:41.993 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:42.139 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-10-12T02:12:42.206 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/2 2023-10-12T02:12:42.441 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:45 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:42.531 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:42.582 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:42.582 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2023-10-12T02:12:42.583 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:42.649 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install rbd-fuse 2023-10-12T02:12:42.903 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2/2 2023-10-12T02:12:42.903 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/2 2023-10-12T02:12:43.201 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-10-12T02:12:43.202 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:43.202 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:43.202 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-10-12T02:12:43.202 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:43.203 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:43.229 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:46 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:43.369 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:43.370 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:43.370 INFO:teuthology.orchestra.run.smithi125.stdout: Package Architecture Version Repository Size 2023-10-12T02:12:43.371 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:43.371 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:43.371 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 87 k 2023-10-12T02:12:43.371 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:43.372 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:43.372 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:43.372 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:12:43.372 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:43.372 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 87 k 2023-10-12T02:12:43.373 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 252 k 2023-10-12T02:12:43.373 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:43.503 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-volume 2023-10-12T02:12:43.555 INFO:teuthology.orchestra.run.smithi125.stdout:rbd-fuse-18.0.0-6684.g9587ffdf.el9.x86_64.rpm 472 kB/s | 87 kB 00:00 2023-10-12T02:12:43.555 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:43.556 INFO:teuthology.orchestra.run.smithi125.stdout:Total 469 kB/s | 87 kB 00:00 2023-10-12T02:12:43.556 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:43.562 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:43.562 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:43.616 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:43.617 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:43.749 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:43.854 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:44.093 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:44 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:44.211 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:44.237 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:44.238 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:44.238 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2023-10-12T02:12:44.238 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:44.239 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:44.239 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-volume noarch 2:18.0.0-6684.g9587ffdf.el9 ceph-noarch 262 k 2023-10-12T02:12:44.239 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:44.239 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:44.240 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:44.240 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:12:44.240 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:44.240 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 262 k 2023-10-12T02:12:44.240 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 1.2 M 2023-10-12T02:12:44.241 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:44.524 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:44.524 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:44.524 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:44.524 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:44.525 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:44.525 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:44.755 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install rbd-mirror 2023-10-12T02:12:44.761 INFO:teuthology.orchestra.run.smithi158.stdout:ceph-volume-18.0.0-6684.g9587ffdf.el9.noarch.rp 502 kB/s | 262 kB 00:00 2023-10-12T02:12:44.762 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:44.762 INFO:teuthology.orchestra.run.smithi158.stdout:Total 500 kB/s | 262 kB 00:00 2023-10-12T02:12:44.763 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:44.770 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:44.770 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:44.803 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:44.803 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:45.014 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:45.080 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:45.095 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:45.095 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:12:45.095 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-12T02:12:45.095 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:45.370 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:48 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:45.518 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:45.519 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:45.519 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2023-10-12T02:12:45.520 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:45.520 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:45.520 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 3.0 M 2023-10-12T02:12:45.520 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:45.520 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:45.521 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:45.521 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:12:45.521 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:45.521 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 3.0 M 2023-10-12T02:12:45.521 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 13 M 2023-10-12T02:12:45.522 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:46.109 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 1/1 2023-10-12T02:12:46.109 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:46.109 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:46.109 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-volume-2:18.0.0-6684.g9587ffdf.el9.noarch 2023-10-12T02:12:46.110 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:46.110 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:46.422 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install librados-devel 2023-10-12T02:12:47.012 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:47 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:47.052 INFO:teuthology.orchestra.run.smithi125.stdout:rbd-mirror-18.0.0-6684.g9587ffdf.el9.x86_64.rpm 2.0 MB/s | 3.0 MB 00:01 2023-10-12T02:12:47.053 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:47.053 INFO:teuthology.orchestra.run.smithi125.stdout:Total 2.0 MB/s | 3.0 MB 00:01 2023-10-12T02:12:47.053 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:47.063 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:47.064 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:47.139 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:47.140 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:47.153 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:47.153 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:47.153 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repo Size 2023-10-12T02:12:47.154 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:47.154 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:47.154 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 127 k 2023-10-12T02:12:47.154 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:47.154 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:47.155 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:47.155 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:12:47.155 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:47.156 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 127 k 2023-10-12T02:12:47.156 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 460 k 2023-10-12T02:12:47.156 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:47.561 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:47.625 INFO:teuthology.orchestra.run.smithi158.stdout:librados-devel-18.0.0-6684.g9587ffdf.el9.x86_64 270 kB/s | 127 kB 00:00 2023-10-12T02:12:47.625 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:47.626 INFO:teuthology.orchestra.run.smithi158.stdout:Total 269 kB/s | 127 kB 00:00 2023-10-12T02:12:47.626 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:47.628 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:47.631 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:47.631 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:47.653 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:47.653 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:12:47.653 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-12T02:12:47.653 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-12T02:12:47.654 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-12T02:12:47.654 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:47.691 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:47.691 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:47.847 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:47.957 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:48.453 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:48.794 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:48.795 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:48.795 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:48.795 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:48.795 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:48.796 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:48.821 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:48.822 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:48.822 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:48.822 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:48.822 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:48.823 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:49.047 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install libcephfs2 2023-10-12T02:12:49.053 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install rbd-nbd 2023-10-12T02:12:49.643 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:49 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:49.645 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:52 ago on Thu 12 Oct 2023 02:09:57 AM UTC. 2023-10-12T02:12:49.736 INFO:teuthology.orchestra.run.smithi158.stdout:Package libcephfs2-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:49.785 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-12T02:12:49.785 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:49.786 INFO:teuthology.orchestra.run.smithi125.stdout: Package Architecture Version Repository Size 2023-10-12T02:12:49.786 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:49.786 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-12T02:12:49.786 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 168 k 2023-10-12T02:12:49.787 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:49.787 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-12T02:12:49.787 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-12T02:12:49.787 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2023-10-12T02:12:49.787 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:49.787 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 168 k 2023-10-12T02:12:49.788 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 533 k 2023-10-12T02:12:49.788 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-12T02:12:49.788 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:49.789 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:49.789 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:49.866 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install libcephfs-devel 2023-10-12T02:12:50.422 INFO:teuthology.orchestra.run.smithi125.stdout:rbd-nbd-18.0.0-6684.g9587ffdf.el9.x86_64.rpm 265 kB/s | 168 kB 00:00 2023-10-12T02:12:50.423 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:50.423 INFO:teuthology.orchestra.run.smithi125.stdout:Total 264 kB/s | 168 kB 00:00 2023-10-12T02:12:50.423 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-12T02:12:50.431 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-12T02:12:50.431 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-12T02:12:50.442 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:50 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:50.493 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-12T02:12:50.494 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-12T02:12:50.582 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:50.583 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:50.583 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repo Size 2023-10-12T02:12:50.583 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:50.583 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:50.584 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 31 k 2023-10-12T02:12:50.584 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:50.584 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:50.584 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:50.584 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:12:50.585 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:50.585 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 31 k 2023-10-12T02:12:50.585 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 139 k 2023-10-12T02:12:50.585 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:50.651 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-12T02:12:50.753 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:50.979 INFO:teuthology.orchestra.run.smithi158.stdout:libcephfs-devel-18.0.0-6684.g9587ffdf.el9.x86_6 79 kB/s | 31 kB 00:00 2023-10-12T02:12:50.980 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:50.980 INFO:teuthology.orchestra.run.smithi158.stdout:Total 78 kB/s | 31 kB 00:00 2023-10-12T02:12:50.980 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:50.983 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:50.983 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:51.015 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:51.015 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:51.127 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:51.241 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:51.252 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:51.564 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:51.564 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:51.564 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-12T02:12:51.565 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:51.565 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:12:51.565 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-12T02:12:51.664 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:51.827 DEBUG:teuthology.parallel:result is None 2023-10-12T02:12:51.933 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:51.933 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:51.934 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:51.934 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:51.934 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:51.934 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:52.174 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install librados2 2023-10-12T02:12:52.753 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:52 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:52.841 INFO:teuthology.orchestra.run.smithi158.stdout:Package librados2-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:52.890 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:52.891 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:52.892 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:52.965 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install librbd1 2023-10-12T02:12:53.548 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:53 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:53.637 INFO:teuthology.orchestra.run.smithi158.stdout:Package librbd1-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:53.686 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:53.687 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:53.687 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:53.761 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install python3-rados 2023-10-12T02:12:54.338 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:54 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:54.428 INFO:teuthology.orchestra.run.smithi158.stdout:Package python3-rados-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:54.477 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:54.478 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:54.478 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:54.569 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install python3-rgw 2023-10-12T02:12:55.159 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:55 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:55.253 INFO:teuthology.orchestra.run.smithi158.stdout:Package python3-rgw-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:55.306 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:55.307 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:55.308 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:55.383 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install python3-cephfs 2023-10-12T02:12:55.991 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:55 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:56.084 INFO:teuthology.orchestra.run.smithi158.stdout:Package python3-cephfs-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:56.136 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:56.137 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:56.137 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:56.213 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install python3-rbd 2023-10-12T02:12:56.790 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:56 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:56.878 INFO:teuthology.orchestra.run.smithi158.stdout:Package python3-rbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 is already installed. 2023-10-12T02:12:56.928 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:56.929 INFO:teuthology.orchestra.run.smithi158.stdout:Nothing to do. 2023-10-12T02:12:56.929 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:57.002 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install rbd-fuse 2023-10-12T02:12:57.583 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:57 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:12:57.721 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:12:57.722 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:57.722 INFO:teuthology.orchestra.run.smithi158.stdout: Package Architecture Version Repository Size 2023-10-12T02:12:57.722 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:57.722 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:12:57.723 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 87 k 2023-10-12T02:12:57.723 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:57.723 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:12:57.723 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:12:57.723 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:12:57.724 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:57.724 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 87 k 2023-10-12T02:12:57.724 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 252 k 2023-10-12T02:12:57.724 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:12:58.008 INFO:teuthology.orchestra.run.smithi158.stdout:rbd-fuse-18.0.0-6684.g9587ffdf.el9.x86_64.rpm 304 kB/s | 87 kB 00:00 2023-10-12T02:12:58.009 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:12:58.009 INFO:teuthology.orchestra.run.smithi158.stdout:Total 303 kB/s | 87 kB 00:00 2023-10-12T02:12:58.009 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:12:58.015 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:12:58.015 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:12:58.070 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:12:58.071 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:12:58.204 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:12:58.305 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:58.760 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:59.067 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:12:59.067 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:59.068 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:12:59.068 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:12:59.068 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:12:59.068 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:12:59.367 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install rbd-mirror 2023-10-12T02:12:59.956 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:02:59 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:13:00.100 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:13:00.101 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:13:00.101 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repo Size 2023-10-12T02:13:00.101 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:13:00.101 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:13:00.101 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 3.0 M 2023-10-12T02:13:00.102 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:00.102 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:13:00.102 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:13:00.102 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:13:00.102 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:00.103 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 3.0 M 2023-10-12T02:13:00.103 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 13 M 2023-10-12T02:13:00.103 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:13:01.336 INFO:teuthology.orchestra.run.smithi158.stdout:rbd-mirror-18.0.0-6684.g9587ffdf.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2023-10-12T02:13:01.336 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:13:01.337 INFO:teuthology.orchestra.run.smithi158.stdout:Total 2.4 MB/s | 3.0 MB 00:01 2023-10-12T02:13:01.337 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:13:01.346 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:13:01.349 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:13:01.419 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:13:01.420 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:13:01.837 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:13:01.895 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:13:01.919 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:13:01.920 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-12T02:13:01.920 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-12T02:13:01.920 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-12T02:13:01.921 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-12T02:13:01.921 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:03.074 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:13:03.075 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:03.075 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:13:03.075 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:13:03.075 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:03.075 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:13:03.361 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install rbd-nbd 2023-10-12T02:13:03.950 INFO:teuthology.orchestra.run.smithi158.stdout:Last metadata expiration check: 0:03:03 ago on Thu 12 Oct 2023 02:10:00 AM UTC. 2023-10-12T02:13:04.088 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2023-10-12T02:13:04.089 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:13:04.090 INFO:teuthology.orchestra.run.smithi158.stdout: Package Architecture Version Repository Size 2023-10-12T02:13:04.090 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:13:04.090 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2023-10-12T02:13:04.090 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd x86_64 2:18.0.0-6684.g9587ffdf.el9 ceph 168 k 2023-10-12T02:13:04.090 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:04.091 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2023-10-12T02:13:04.091 INFO:teuthology.orchestra.run.smithi158.stdout:================================================================================ 2023-10-12T02:13:04.091 INFO:teuthology.orchestra.run.smithi158.stdout:Install 1 Package 2023-10-12T02:13:04.091 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:04.092 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 168 k 2023-10-12T02:13:04.092 INFO:teuthology.orchestra.run.smithi158.stdout:Installed size: 533 k 2023-10-12T02:13:04.092 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2023-10-12T02:13:04.507 INFO:teuthology.orchestra.run.smithi158.stdout:rbd-nbd-18.0.0-6684.g9587ffdf.el9.x86_64.rpm 405 kB/s | 168 kB 00:00 2023-10-12T02:13:04.507 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2023-10-12T02:13:04.508 INFO:teuthology.orchestra.run.smithi158.stdout:Total 403 kB/s | 168 kB 00:00 2023-10-12T02:13:04.508 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2023-10-12T02:13:04.515 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2023-10-12T02:13:04.515 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2023-10-12T02:13:04.576 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2023-10-12T02:13:04.576 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2023-10-12T02:13:04.742 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2023-10-12T02:13:04.848 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:13:05.303 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:13:05.613 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 1/1 2023-10-12T02:13:05.614 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:05.614 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2023-10-12T02:13:05.614 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd-2:18.0.0-6684.g9587ffdf.el9.x86_64 2023-10-12T02:13:05.615 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:05.615 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2023-10-12T02:13:05.923 DEBUG:teuthology.parallel:result is None 2023-10-12T02:13:05.924 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 2023-10-12T02:13:06.061 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-12T02:13:06.086 INFO:teuthology.orchestra.run.smithi125.stdout:18.0.0-6684.g9587ffdf.el9 2023-10-12T02:13:06.086 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6684.g9587ffdf.el9 2023-10-12T02:13:06.087 INFO:teuthology.task.install:The correct ceph version 18.0.0-6684.g9587ffdf is installed. 2023-10-12T02:13:06.087 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 2023-10-12T02:13:06.228 DEBUG:teuthology.orchestra.run.smithi158:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-12T02:13:06.249 INFO:teuthology.orchestra.run.smithi158.stdout:18.0.0-6684.g9587ffdf.el9 2023-10-12T02:13:06.250 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6684.g9587ffdf.el9 2023-10-12T02:13:06.250 INFO:teuthology.task.install:The correct ceph version 18.0.0-6684.g9587ffdf is installed. 2023-10-12T02:13:06.250 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-12T02:13:06.251 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:06.251 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-12T02:13:06.285 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:06.285 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-12T02:13:06.316 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-12T02:13:06.316 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:06.317 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/daemon-helper 2023-10-12T02:13:06.356 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-12T02:13:06.426 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:06.426 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/daemon-helper 2023-10-12T02:13:06.452 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-12T02:13:06.519 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-12T02:13:06.519 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:06.520 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-12T02:13:06.549 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-12T02:13:06.619 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:06.619 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-12T02:13:06.647 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-12T02:13:06.713 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-12T02:13:06.713 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:06.714 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/stdin-killer 2023-10-12T02:13:06.743 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-12T02:13:06.811 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:06.811 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/stdin-killer 2023-10-12T02:13:06.838 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-12T02:13:06.905 INFO:teuthology.run_tasks:Running task ceph... 2023-10-12T02:13:06.973 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-12T02:13:06.974 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 777 /var/log/ceph 2023-10-12T02:13:06.976 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 777 /var/log/ceph 2023-10-12T02:13:07.003 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-12T02:13:07.003 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-12T02:13:07.044 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-12T02:13:07.070 INFO:tasks.ceph:Creating extra log directories... 2023-10-12T02:13:07.071 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-12T02:13:07.111 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-12T02:13:07.147 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-12T02:13:07.147 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-12T02:13:07.148 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367', 'branch': 'wip-yingxin-crimson-osd-crosscore-pg-submission', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': 'yingxin.cheng@intel.com', 'first_in_suite': False, 'job_id': '7422367', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yingxin-crimson-osd-crosscore-pg-submission'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5'}}, 'workunit': {'branch': 'wip-yingxin-crimson-osd-crosscore-pg-submission', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5'}}, 'owner': 'scheduled_yingxin@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-yingxin-crimson-osd-crosscore-pg-submission', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5', 'targets': {'smithi125.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN73DjrdE1g/zWHtmfxoMBCa2KEZsi38QXldujS7u0Zww7nZEB2UwjO8NtmyYYBxfjV9JEweM2TVFCIb8a4Mc+8=', 'smithi158.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCBwCQt/Ud2a8XaZeiwNWB9DezxlH0sYlAKXUvpN15HLpscSYamtBdB1EmEAaFCHN7visu6WEigz8M/5Vw1NYI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-12_01:35:43', 'tube': 'smithi', 'user': 'yingxin', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-12T02:13:07.148 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-12T02:13:07.179 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-12T02:13:07.194 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m0777 -- /var/run/ceph 2023-10-12T02:13:07.234 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -d -m0777 -- /var/run/ceph 2023-10-12T02:13:07.261 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:07.261 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/scratch_devs of=/dev/stdout 2023-10-12T02:13:07.314 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-12T02:13:07.314 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_1 2023-10-12T02:13:07.368 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-12T02:13:07.369 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:07.369 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-10-12T02:13:07.369 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:07.369 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:07.370 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-12 02:12:47.893913194 +0000 2023-10-12T02:13:07.370 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-12 02:09:09.868269043 +0000 2023-10-12T02:13:07.370 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-12 02:09:09.868269043 +0000 2023-10-12T02:13:07.370 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-12 02:09:09.868269043 +0000 2023-10-12T02:13:07.370 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-12T02:13:07.434 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-12T02:13:07.434 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-12T02:13:07.434 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000197315 s, 2.6 MB/s 2023-10-12T02:13:07.436 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-12T02:13:07.491 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_2 2023-10-12T02:13:07.546 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-12T02:13:07.546 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:07.547 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 684 Links: 1 2023-10-12T02:13:07.547 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:07.547 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:07.547 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-12 02:12:47.893913194 +0000 2023-10-12T02:13:07.547 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-12 02:09:10.369260027 +0000 2023-10-12T02:13:07.548 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-12 02:09:10.369260027 +0000 2023-10-12T02:13:07.548 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-12 02:09:10.369260027 +0000 2023-10-12T02:13:07.548 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-12T02:13:07.611 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-12T02:13:07.611 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-12T02:13:07.611 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000202445 s, 2.5 MB/s 2023-10-12T02:13:07.612 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-12T02:13:07.668 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_3 2023-10-12T02:13:07.722 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-12T02:13:07.722 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:07.722 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 700 Links: 1 2023-10-12T02:13:07.723 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:07.723 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:07.723 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-12 02:12:47.894913176 +0000 2023-10-12T02:13:07.725 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-12 02:09:10.857251245 +0000 2023-10-12T02:13:07.725 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-12 02:09:10.857251245 +0000 2023-10-12T02:13:07.725 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-12 02:09:10.857251245 +0000 2023-10-12T02:13:07.726 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-12T02:13:07.786 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-12T02:13:07.787 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-12T02:13:07.787 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000167717 s, 3.1 MB/s 2023-10-12T02:13:07.788 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-12T02:13:07.844 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_4 2023-10-12T02:13:07.898 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-12T02:13:07.899 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:07.899 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-10-12T02:13:07.899 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:07.899 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:07.900 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-12 02:12:47.894913176 +0000 2023-10-12T02:13:07.900 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-12 02:09:11.378241870 +0000 2023-10-12T02:13:07.900 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-12 02:09:11.378241870 +0000 2023-10-12T02:13:07.900 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-12 02:09:11.378241870 +0000 2023-10-12T02:13:07.901 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-12T02:13:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-12T02:13:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-12T02:13:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.00024806 s, 2.1 MB/s 2023-10-12T02:13:07.967 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-12T02:13:08.022 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-10-12T02:13:08.022 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:08.023 DEBUG:teuthology.orchestra.run.smithi158:> dd if=/scratch_devs of=/dev/stdout 2023-10-12T02:13:08.039 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-12T02:13:08.040 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_1 2023-10-12T02:13:08.095 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-12T02:13:08.095 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:08.096 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-10-12T02:13:08.096 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:08.096 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:08.096 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-10-12 02:13:02.151068317 +0000 2023-10-12T02:13:08.096 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-10-12 02:09:01.334228475 +0000 2023-10-12T02:13:08.097 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-10-12 02:09:01.334228475 +0000 2023-10-12T02:13:08.097 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2023-10-12 02:09:01.334228475 +0000 2023-10-12T02:13:08.097 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-12T02:13:08.162 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-10-12T02:13:08.162 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-10-12T02:13:08.163 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 8.763e-05 s, 5.8 MB/s 2023-10-12T02:13:08.164 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-12T02:13:08.221 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_2 2023-10-12T02:13:08.277 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-12T02:13:08.278 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:08.278 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-10-12T02:13:08.278 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:08.278 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:08.279 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-10-12 02:13:02.151068317 +0000 2023-10-12T02:13:08.279 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-10-12 02:09:01.817223182 +0000 2023-10-12T02:13:08.279 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-10-12 02:09:01.817223182 +0000 2023-10-12T02:13:08.279 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2023-10-12 02:09:01.817223182 +0000 2023-10-12T02:13:08.279 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-12T02:13:08.343 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-10-12T02:13:08.344 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-10-12T02:13:08.344 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 9.9239e-05 s, 5.2 MB/s 2023-10-12T02:13:08.345 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-12T02:13:08.401 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_3 2023-10-12T02:13:08.456 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-10-12T02:13:08.457 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-10-12 02:13:02.152068306 +0000 2023-10-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-10-12 02:09:02.311217769 +0000 2023-10-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-10-12 02:09:02.311217769 +0000 2023-10-12T02:13:08.458 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2023-10-12 02:09:02.311217769 +0000 2023-10-12T02:13:08.459 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-12T02:13:08.522 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-10-12T02:13:08.522 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-10-12T02:13:08.522 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000104282 s, 4.9 MB/s 2023-10-12T02:13:08.523 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-12T02:13:08.579 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_4 2023-10-12T02:13:08.634 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-12T02:13:08.634 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-12T02:13:08.634 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 722 Links: 1 2023-10-12T02:13:08.635 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-12T02:13:08.635 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2023-10-12T02:13:08.635 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2023-10-12 02:13:02.152068306 +0000 2023-10-12T02:13:08.635 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2023-10-12 02:09:02.821212180 +0000 2023-10-12T02:13:08.635 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2023-10-12 02:09:02.821212180 +0000 2023-10-12T02:13:08.636 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2023-10-12 02:09:02.821212180 +0000 2023-10-12T02:13:08.636 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-12T02:13:08.699 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2023-10-12T02:13:08.699 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2023-10-12T02:13:08.700 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000100411 s, 5.1 MB/s 2023-10-12T02:13:08.702 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-12T02:13:08.758 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-10-12T02:13:08.759 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi125.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi158.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-10-12T02:13:08.759 INFO:tasks.ceph:Generating config... 2023-10-12T02:13:08.760 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-10-12T02:13:08.761 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-10-12T02:13:08.761 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-10-12T02:13:08.761 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-10-12T02:13:08.762 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-10-12T02:13:08.762 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-12T02:13:08.762 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-12T02:13:08.762 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-12T02:13:08.763 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2023-10-12T02:13:08.763 INFO:tasks.ceph:[mon] osd pool default crimson = True 2023-10-12T02:13:08.763 INFO:tasks.ceph:[osd] debug monc = 20 2023-10-12T02:13:08.763 INFO:tasks.ceph:[osd] crimson seastar smp = 3 2023-10-12T02:13:08.764 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-12T02:13:08.764 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-12T02:13:08.764 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-10-12T02:13:08.765 INFO:tasks.ceph:[osd] osd class default list = * 2023-10-12T02:13:08.765 INFO:tasks.ceph:[osd] osd class load list = * 2023-10-12T02:13:08.765 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-10-12T02:13:08.765 INFO:tasks.ceph:[client] debug ms = 1 2023-10-12T02:13:08.766 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2023-10-12T02:13:08.766 INFO:tasks.ceph:[global] ms client mode = crc 2023-10-12T02:13:08.766 INFO:tasks.ceph:[global] ms cluster mode = crc 2023-10-12T02:13:08.767 INFO:tasks.ceph:[global] ms mon client mode = crc 2023-10-12T02:13:08.767 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2023-10-12T02:13:08.767 INFO:tasks.ceph:[global] ms mon service mode = crc 2023-10-12T02:13:08.767 INFO:tasks.ceph:[global] ms service mode = crc 2023-10-12T02:13:08.768 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-12T02:13:08.768 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-12T02:13:08.768 INFO:tasks.ceph:Setting up mon.a... 2023-10-12T02:13:08.769 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-12T02:13:08.833 INFO:teuthology.orchestra.run.smithi125.stdout:creating /etc/ceph/ceph.keyring 2023-10-12T02:13:08.836 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-12T02:13:08.889 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-12T02:13:08.916 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.125')] 2023-10-12T02:13:08.916 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.125', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}} 2023-10-12T02:13:08.917 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:08.918 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-12T02:13:08.971 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.125 --print /home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:09.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:09.054+0000 7f62f1c39e80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:09.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:09.054+0000 7f62f1c39e80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:09.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:09.054+0000 7f62f1c39e80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:09.058 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:09.058 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: generated fsid bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:09.058 INFO:teuthology.orchestra.run.smithi125.stdout:setting min_mon_release = pacific 2023-10-12T02:13:09.058 INFO:teuthology.orchestra.run.smithi125.stdout:epoch 0 2023-10-12T02:13:09.059 INFO:teuthology.orchestra.run.smithi125.stdout:fsid bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:09.059 INFO:teuthology.orchestra.run.smithi125.stdout:last_changed 2023-10-12T02:13:09.055908+0000 2023-10-12T02:13:09.059 INFO:teuthology.orchestra.run.smithi125.stdout:created 2023-10-12T02:13:09.055908+0000 2023-10-12T02:13:09.059 INFO:teuthology.orchestra.run.smithi125.stdout:min_mon_release 16 (pacific) 2023-10-12T02:13:09.059 INFO:teuthology.orchestra.run.smithi125.stdout:election_strategy: 1 2023-10-12T02:13:09.060 INFO:teuthology.orchestra.run.smithi125.stdout:0: [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] mon.a 2023-10-12T02:13:09.060 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-10-12T02:13:09.061 DEBUG:teuthology.orchestra.run.smithi125:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-12T02:13:09.112 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bd5090fa-2483-41ce-a0be-4321207eb2da... 2023-10-12T02:13:09.113 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-12T02:13:09.154 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-12T02:13:09.201 INFO:teuthology.orchestra.run.smithi125.stdout:[global] 2023-10-12T02:13:09.202 INFO:teuthology.orchestra.run.smithi125.stdout: chdir = "" 2023-10-12T02:13:09.202 INFO:teuthology.orchestra.run.smithi125.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-12T02:13:09.202 INFO:teuthology.orchestra.run.smithi125.stdout: auth supported = cephx 2023-10-12T02:13:09.202 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.203 INFO:teuthology.orchestra.run.smithi125.stdout: filestore xattr use omap = true 2023-10-12T02:13:09.203 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.203 INFO:teuthology.orchestra.run.smithi125.stdout: mon clock drift allowed = 1.000 2023-10-12T02:13:09.203 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.203 INFO:teuthology.orchestra.run.smithi125.stdout: osd crush chooseleaf type = 0 2023-10-12T02:13:09.204 INFO:teuthology.orchestra.run.smithi125.stdout: auth debug = true 2023-10-12T02:13:09.204 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.204 INFO:teuthology.orchestra.run.smithi125.stdout: ms die on old message = true 2023-10-12T02:13:09.204 INFO:teuthology.orchestra.run.smithi125.stdout: ms die on bug = true 2023-10-12T02:13:09.205 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.205 INFO:teuthology.orchestra.run.smithi125.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-12T02:13:09.205 INFO:teuthology.orchestra.run.smithi125.stdout: mon pg warn max object skew = 0 2023-10-12T02:13:09.205 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.205 INFO:teuthology.orchestra.run.smithi125.stdout: # disable pg_autoscaler by default for new pools 2023-10-12T02:13:09.206 INFO:teuthology.orchestra.run.smithi125.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-12T02:13:09.206 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.206 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default size = 2 2023-10-12T02:13:09.206 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.206 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd allow primary affinity = true 2023-10-12T02:13:09.207 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd allow pg remap = true 2023-10-12T02:13:09.207 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on legacy crush tunables = false 2023-10-12T02:13:09.207 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on crush straw calc version zero = false 2023-10-12T02:13:09.207 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on no sortbitwise = false 2023-10-12T02:13:09.207 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on osd down out interval zero = false 2023-10-12T02:13:09.207 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on too few osds = false 2023-10-12T02:13:09.208 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-12T02:13:09.208 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-12T02:13:09.208 INFO:teuthology.orchestra.run.smithi125.stdout: mon_allow_pool_size_one = true 2023-10-12T02:13:09.208 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.208 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-12T02:13:09.209 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.209 INFO:teuthology.orchestra.run.smithi125.stdout: osd default data pool replay window = 5 2023-10-12T02:13:09.209 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.209 INFO:teuthology.orchestra.run.smithi125.stdout: mon allow pool delete = true 2023-10-12T02:13:09.209 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.210 INFO:teuthology.orchestra.run.smithi125.stdout: mon cluster log file level = debug 2023-10-12T02:13:09.210 INFO:teuthology.orchestra.run.smithi125.stdout: debug asserts on shutdown = true 2023-10-12T02:13:09.210 INFO:teuthology.orchestra.run.smithi125.stdout: mon health detail to clog = false 2023-10-12T02:13:09.210 INFO:teuthology.orchestra.run.smithi125.stdout: mon host = 172.21.15.125 2023-10-12T02:13:09.210 INFO:teuthology.orchestra.run.smithi125.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-10-12T02:13:09.211 INFO:teuthology.orchestra.run.smithi125.stdout: ms client mode = crc 2023-10-12T02:13:09.211 INFO:teuthology.orchestra.run.smithi125.stdout: ms cluster mode = crc 2023-10-12T02:13:09.211 INFO:teuthology.orchestra.run.smithi125.stdout: ms mon client mode = crc 2023-10-12T02:13:09.211 INFO:teuthology.orchestra.run.smithi125.stdout: ms mon cluster mode = crc 2023-10-12T02:13:09.211 INFO:teuthology.orchestra.run.smithi125.stdout: ms mon service mode = crc 2023-10-12T02:13:09.212 INFO:teuthology.orchestra.run.smithi125.stdout: ms service mode = crc 2023-10-12T02:13:09.212 INFO:teuthology.orchestra.run.smithi125.stdout: fsid = bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:09.212 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.212 INFO:teuthology.orchestra.run.smithi125.stdout:[osd] 2023-10-12T02:13:09.212 INFO:teuthology.orchestra.run.smithi125.stdout: osd journal size = 100 2023-10-12T02:13:09.213 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.213 INFO:teuthology.orchestra.run.smithi125.stdout: osd scrub load threshold = 5.0 2023-10-12T02:13:09.213 INFO:teuthology.orchestra.run.smithi125.stdout: osd scrub max interval = 600 2023-10-12T02:13:09.213 INFO:teuthology.orchestra.run.smithi125.stdout: osd mclock profile = high_recovery_ops 2023-10-12T02:13:09.213 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.214 INFO:teuthology.orchestra.run.smithi125.stdout: osd recover clone overlap = true 2023-10-12T02:13:09.214 INFO:teuthology.orchestra.run.smithi125.stdout: osd recovery max chunk = 1048576 2023-10-12T02:13:09.214 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.214 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug shutdown = true 2023-10-12T02:13:09.215 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug op order = true 2023-10-12T02:13:09.215 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug verify stray on activate = true 2023-10-12T02:13:09.215 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.215 INFO:teuthology.orchestra.run.smithi125.stdout: osd open classes on start = true 2023-10-12T02:13:09.215 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug pg log writeout = true 2023-10-12T02:13:09.216 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.216 INFO:teuthology.orchestra.run.smithi125.stdout: osd deep scrub update digest min age = 30 2023-10-12T02:13:09.216 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.216 INFO:teuthology.orchestra.run.smithi125.stdout: osd map max advance = 10 2023-10-12T02:13:09.216 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.217 INFO:teuthology.orchestra.run.smithi125.stdout: journal zero on create = true 2023-10-12T02:13:09.217 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.217 INFO:teuthology.orchestra.run.smithi125.stdout: filestore ondisk finisher threads = 3 2023-10-12T02:13:09.217 INFO:teuthology.orchestra.run.smithi125.stdout: filestore apply finisher threads = 3 2023-10-12T02:13:09.218 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.218 INFO:teuthology.orchestra.run.smithi125.stdout: bdev debug aio = true 2023-10-12T02:13:09.219 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug misdirected ops = true 2023-10-12T02:13:09.219 INFO:teuthology.orchestra.run.smithi125.stdout: debug monc = 20 2023-10-12T02:13:09.219 INFO:teuthology.orchestra.run.smithi125.stdout: crimson seastar smp = 3 2023-10-12T02:13:09.219 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2023-10-12T02:13:09.220 INFO:teuthology.orchestra.run.smithi125.stdout: debug osd = 20 2023-10-12T02:13:09.220 INFO:teuthology.orchestra.run.smithi125.stdout: osd blocked scrub grace period = 3600 2023-10-12T02:13:09.220 INFO:teuthology.orchestra.run.smithi125.stdout: osd class default list = * 2023-10-12T02:13:09.220 INFO:teuthology.orchestra.run.smithi125.stdout: osd class load list = * 2023-10-12T02:13:09.220 INFO:teuthology.orchestra.run.smithi125.stdout: osd shutdown pgref assert = True 2023-10-12T02:13:09.220 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.221 INFO:teuthology.orchestra.run.smithi125.stdout:[mgr] 2023-10-12T02:13:09.221 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2023-10-12T02:13:09.221 INFO:teuthology.orchestra.run.smithi125.stdout: debug mgr = 20 2023-10-12T02:13:09.221 INFO:teuthology.orchestra.run.smithi125.stdout: debug mon = 20 2023-10-12T02:13:09.221 INFO:teuthology.orchestra.run.smithi125.stdout: debug auth = 20 2023-10-12T02:13:09.222 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min pgs per osd = 4 2023-10-12T02:13:09.222 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min bytes per osd = 10 2023-10-12T02:13:09.222 INFO:teuthology.orchestra.run.smithi125.stdout: mgr/telemetry/nag = false 2023-10-12T02:13:09.222 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.223 INFO:teuthology.orchestra.run.smithi125.stdout:[mon] 2023-10-12T02:13:09.223 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2023-10-12T02:13:09.223 INFO:teuthology.orchestra.run.smithi125.stdout: debug mon = 20 2023-10-12T02:13:09.223 INFO:teuthology.orchestra.run.smithi125.stdout: debug paxos = 20 2023-10-12T02:13:09.223 INFO:teuthology.orchestra.run.smithi125.stdout: debug auth = 20 2023-10-12T02:13:09.223 INFO:teuthology.orchestra.run.smithi125.stdout: mon data avail warn = 5 2023-10-12T02:13:09.224 INFO:teuthology.orchestra.run.smithi125.stdout: mon mgr mkfs grace = 240 2023-10-12T02:13:09.224 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min pgs per osd = 4 2023-10-12T02:13:09.224 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd reporter subtree level = osd 2023-10-12T02:13:09.225 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd prime pg temp = true 2023-10-12T02:13:09.225 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min bytes per osd = 10 2023-10-12T02:13:09.225 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.225 INFO:teuthology.orchestra.run.smithi125.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-12T02:13:09.226 INFO:teuthology.orchestra.run.smithi125.stdout: auth mon ticket ttl = 660 # 11m 2023-10-12T02:13:09.226 INFO:teuthology.orchestra.run.smithi125.stdout: auth service ticket ttl = 240 # 4m 2023-10-12T02:13:09.226 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.226 INFO:teuthology.orchestra.run.smithi125.stdout: # don't complain about insecure global_id in the test suite 2023-10-12T02:13:09.227 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-12T02:13:09.227 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-12T02:13:09.227 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.227 INFO:teuthology.orchestra.run.smithi125.stdout: # 1m isn't quite enough 2023-10-12T02:13:09.227 INFO:teuthology.orchestra.run.smithi125.stdout: mon_down_mkfs_grace = 2m 2023-10-12T02:13:09.228 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.228 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_filestore_osds = false 2023-10-12T02:13:09.228 INFO:teuthology.orchestra.run.smithi125.stdout: mon min osdmap epochs = 50 2023-10-12T02:13:09.228 INFO:teuthology.orchestra.run.smithi125.stdout: mon osdmap full prune interval = 2 2023-10-12T02:13:09.228 INFO:teuthology.orchestra.run.smithi125.stdout: mon osdmap full prune min = 15 2023-10-12T02:13:09.229 INFO:teuthology.orchestra.run.smithi125.stdout: mon osdmap full prune txsize = 2 2023-10-12T02:13:09.229 INFO:teuthology.orchestra.run.smithi125.stdout: paxos service trim min = 10 2023-10-12T02:13:09.229 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on pool no app = False 2023-10-12T02:13:09.229 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default crimson = True 2023-10-12T02:13:09.229 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:09.230 INFO:teuthology.orchestra.run.smithi125.stdout:[client] 2023-10-12T02:13:09.230 INFO:teuthology.orchestra.run.smithi125.stdout: rgw cache enabled = true 2023-10-12T02:13:09.230 INFO:teuthology.orchestra.run.smithi125.stdout: rgw enable ops log = true 2023-10-12T02:13:09.230 INFO:teuthology.orchestra.run.smithi125.stdout: rgw enable usage log = true 2023-10-12T02:13:09.231 INFO:teuthology.orchestra.run.smithi125.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-12T02:13:09.231 INFO:teuthology.orchestra.run.smithi125.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-12T02:13:09.231 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2023-10-12T02:13:09.231 INFO:teuthology.orchestra.run.smithi125.stdout:[mon.a] 2023-10-12T02:13:09.233 INFO:teuthology.orchestra.run.smithi158.stdout:[global] 2023-10-12T02:13:09.233 INFO:teuthology.orchestra.run.smithi158.stdout: chdir = "" 2023-10-12T02:13:09.233 INFO:teuthology.orchestra.run.smithi158.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-12T02:13:09.233 INFO:teuthology.orchestra.run.smithi158.stdout: auth supported = cephx 2023-10-12T02:13:09.234 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.234 INFO:teuthology.orchestra.run.smithi158.stdout: filestore xattr use omap = true 2023-10-12T02:13:09.234 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.234 INFO:teuthology.orchestra.run.smithi158.stdout: mon clock drift allowed = 1.000 2023-10-12T02:13:09.234 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.235 INFO:teuthology.orchestra.run.smithi158.stdout: osd crush chooseleaf type = 0 2023-10-12T02:13:09.235 INFO:teuthology.orchestra.run.smithi158.stdout: auth debug = true 2023-10-12T02:13:09.235 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.235 INFO:teuthology.orchestra.run.smithi158.stdout: ms die on old message = true 2023-10-12T02:13:09.235 INFO:teuthology.orchestra.run.smithi158.stdout: ms die on bug = true 2023-10-12T02:13:09.236 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.236 INFO:teuthology.orchestra.run.smithi158.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-12T02:13:09.236 INFO:teuthology.orchestra.run.smithi158.stdout: mon pg warn max object skew = 0 2023-10-12T02:13:09.236 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.236 INFO:teuthology.orchestra.run.smithi158.stdout: # disable pg_autoscaler by default for new pools 2023-10-12T02:13:09.237 INFO:teuthology.orchestra.run.smithi158.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-12T02:13:09.237 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.237 INFO:teuthology.orchestra.run.smithi158.stdout: osd pool default size = 2 2023-10-12T02:13:09.237 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.238 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd allow primary affinity = true 2023-10-12T02:13:09.238 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd allow pg remap = true 2023-10-12T02:13:09.238 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on legacy crush tunables = false 2023-10-12T02:13:09.238 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on crush straw calc version zero = false 2023-10-12T02:13:09.238 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on no sortbitwise = false 2023-10-12T02:13:09.239 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on osd down out interval zero = false 2023-10-12T02:13:09.239 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on too few osds = false 2023-10-12T02:13:09.239 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-12T02:13:09.239 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-12T02:13:09.239 INFO:teuthology.orchestra.run.smithi158.stdout: mon_allow_pool_size_one = true 2023-10-12T02:13:09.240 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.240 INFO:teuthology.orchestra.run.smithi158.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-12T02:13:09.240 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.240 INFO:teuthology.orchestra.run.smithi158.stdout: osd default data pool replay window = 5 2023-10-12T02:13:09.241 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.241 INFO:teuthology.orchestra.run.smithi158.stdout: mon allow pool delete = true 2023-10-12T02:13:09.241 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.241 INFO:teuthology.orchestra.run.smithi158.stdout: mon cluster log file level = debug 2023-10-12T02:13:09.241 INFO:teuthology.orchestra.run.smithi158.stdout: debug asserts on shutdown = true 2023-10-12T02:13:09.242 INFO:teuthology.orchestra.run.smithi158.stdout: mon health detail to clog = false 2023-10-12T02:13:09.242 INFO:teuthology.orchestra.run.smithi158.stdout: mon host = 172.21.15.125 2023-10-12T02:13:09.242 INFO:teuthology.orchestra.run.smithi158.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-10-12T02:13:09.242 INFO:teuthology.orchestra.run.smithi158.stdout: ms client mode = crc 2023-10-12T02:13:09.242 INFO:teuthology.orchestra.run.smithi158.stdout: ms cluster mode = crc 2023-10-12T02:13:09.243 INFO:teuthology.orchestra.run.smithi158.stdout: ms mon client mode = crc 2023-10-12T02:13:09.243 INFO:teuthology.orchestra.run.smithi158.stdout: ms mon cluster mode = crc 2023-10-12T02:13:09.243 INFO:teuthology.orchestra.run.smithi158.stdout: ms mon service mode = crc 2023-10-12T02:13:09.243 INFO:teuthology.orchestra.run.smithi158.stdout: ms service mode = crc 2023-10-12T02:13:09.243 INFO:teuthology.orchestra.run.smithi158.stdout: fsid = bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:09.243 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.244 INFO:teuthology.orchestra.run.smithi158.stdout:[osd] 2023-10-12T02:13:09.244 INFO:teuthology.orchestra.run.smithi158.stdout: osd journal size = 100 2023-10-12T02:13:09.244 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.244 INFO:teuthology.orchestra.run.smithi158.stdout: osd scrub load threshold = 5.0 2023-10-12T02:13:09.244 INFO:teuthology.orchestra.run.smithi158.stdout: osd scrub max interval = 600 2023-10-12T02:13:09.245 INFO:teuthology.orchestra.run.smithi158.stdout: osd mclock profile = high_recovery_ops 2023-10-12T02:13:09.245 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.245 INFO:teuthology.orchestra.run.smithi158.stdout: osd recover clone overlap = true 2023-10-12T02:13:09.245 INFO:teuthology.orchestra.run.smithi158.stdout: osd recovery max chunk = 1048576 2023-10-12T02:13:09.245 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.246 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug shutdown = true 2023-10-12T02:13:09.246 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug op order = true 2023-10-12T02:13:09.246 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug verify stray on activate = true 2023-10-12T02:13:09.246 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.246 INFO:teuthology.orchestra.run.smithi158.stdout: osd open classes on start = true 2023-10-12T02:13:09.247 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug pg log writeout = true 2023-10-12T02:13:09.247 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.247 INFO:teuthology.orchestra.run.smithi158.stdout: osd deep scrub update digest min age = 30 2023-10-12T02:13:09.247 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.248 INFO:teuthology.orchestra.run.smithi158.stdout: osd map max advance = 10 2023-10-12T02:13:09.248 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.248 INFO:teuthology.orchestra.run.smithi158.stdout: journal zero on create = true 2023-10-12T02:13:09.248 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.248 INFO:teuthology.orchestra.run.smithi158.stdout: filestore ondisk finisher threads = 3 2023-10-12T02:13:09.248 INFO:teuthology.orchestra.run.smithi158.stdout: filestore apply finisher threads = 3 2023-10-12T02:13:09.249 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.249 INFO:teuthology.orchestra.run.smithi158.stdout: bdev debug aio = true 2023-10-12T02:13:09.249 INFO:teuthology.orchestra.run.smithi158.stdout: osd debug misdirected ops = true 2023-10-12T02:13:09.249 INFO:teuthology.orchestra.run.smithi158.stdout: debug monc = 20 2023-10-12T02:13:09.249 INFO:teuthology.orchestra.run.smithi158.stdout: crimson seastar smp = 3 2023-10-12T02:13:09.249 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-10-12T02:13:09.250 INFO:teuthology.orchestra.run.smithi158.stdout: debug osd = 20 2023-10-12T02:13:09.250 INFO:teuthology.orchestra.run.smithi158.stdout: osd blocked scrub grace period = 3600 2023-10-12T02:13:09.250 INFO:teuthology.orchestra.run.smithi158.stdout: osd class default list = * 2023-10-12T02:13:09.250 INFO:teuthology.orchestra.run.smithi158.stdout: osd class load list = * 2023-10-12T02:13:09.250 INFO:teuthology.orchestra.run.smithi158.stdout: osd shutdown pgref assert = True 2023-10-12T02:13:09.251 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.251 INFO:teuthology.orchestra.run.smithi158.stdout:[mgr] 2023-10-12T02:13:09.251 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-10-12T02:13:09.251 INFO:teuthology.orchestra.run.smithi158.stdout: debug mgr = 20 2023-10-12T02:13:09.251 INFO:teuthology.orchestra.run.smithi158.stdout: debug mon = 20 2023-10-12T02:13:09.252 INFO:teuthology.orchestra.run.smithi158.stdout: debug auth = 20 2023-10-12T02:13:09.252 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min pgs per osd = 4 2023-10-12T02:13:09.252 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min bytes per osd = 10 2023-10-12T02:13:09.252 INFO:teuthology.orchestra.run.smithi158.stdout: mgr/telemetry/nag = false 2023-10-12T02:13:09.252 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.252 INFO:teuthology.orchestra.run.smithi158.stdout:[mon] 2023-10-12T02:13:09.253 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-10-12T02:13:09.253 INFO:teuthology.orchestra.run.smithi158.stdout: debug mon = 20 2023-10-12T02:13:09.253 INFO:teuthology.orchestra.run.smithi158.stdout: debug paxos = 20 2023-10-12T02:13:09.253 INFO:teuthology.orchestra.run.smithi158.stdout: debug auth = 20 2023-10-12T02:13:09.253 INFO:teuthology.orchestra.run.smithi158.stdout: mon data avail warn = 5 2023-10-12T02:13:09.254 INFO:teuthology.orchestra.run.smithi158.stdout: mon mgr mkfs grace = 240 2023-10-12T02:13:09.254 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min pgs per osd = 4 2023-10-12T02:13:09.254 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd reporter subtree level = osd 2023-10-12T02:13:09.254 INFO:teuthology.orchestra.run.smithi158.stdout: mon osd prime pg temp = true 2023-10-12T02:13:09.255 INFO:teuthology.orchestra.run.smithi158.stdout: mon reweight min bytes per osd = 10 2023-10-12T02:13:09.255 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.255 INFO:teuthology.orchestra.run.smithi158.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-12T02:13:09.255 INFO:teuthology.orchestra.run.smithi158.stdout: auth mon ticket ttl = 660 # 11m 2023-10-12T02:13:09.255 INFO:teuthology.orchestra.run.smithi158.stdout: auth service ticket ttl = 240 # 4m 2023-10-12T02:13:09.256 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.256 INFO:teuthology.orchestra.run.smithi158.stdout: # don't complain about insecure global_id in the test suite 2023-10-12T02:13:09.256 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-12T02:13:09.256 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-12T02:13:09.256 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.257 INFO:teuthology.orchestra.run.smithi158.stdout: # 1m isn't quite enough 2023-10-12T02:13:09.257 INFO:teuthology.orchestra.run.smithi158.stdout: mon_down_mkfs_grace = 2m 2023-10-12T02:13:09.257 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.257 INFO:teuthology.orchestra.run.smithi158.stdout: mon_warn_on_filestore_osds = false 2023-10-12T02:13:09.257 INFO:teuthology.orchestra.run.smithi158.stdout: mon min osdmap epochs = 50 2023-10-12T02:13:09.257 INFO:teuthology.orchestra.run.smithi158.stdout: mon osdmap full prune interval = 2 2023-10-12T02:13:09.258 INFO:teuthology.orchestra.run.smithi158.stdout: mon osdmap full prune min = 15 2023-10-12T02:13:09.258 INFO:teuthology.orchestra.run.smithi158.stdout: mon osdmap full prune txsize = 2 2023-10-12T02:13:09.258 INFO:teuthology.orchestra.run.smithi158.stdout: paxos service trim min = 10 2023-10-12T02:13:09.258 INFO:teuthology.orchestra.run.smithi158.stdout: mon warn on pool no app = False 2023-10-12T02:13:09.258 INFO:teuthology.orchestra.run.smithi158.stdout: osd pool default crimson = True 2023-10-12T02:13:09.259 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-10-12T02:13:09.259 INFO:teuthology.orchestra.run.smithi158.stdout:[client] 2023-10-12T02:13:09.259 INFO:teuthology.orchestra.run.smithi158.stdout: rgw cache enabled = true 2023-10-12T02:13:09.259 INFO:teuthology.orchestra.run.smithi158.stdout: rgw enable ops log = true 2023-10-12T02:13:09.259 INFO:teuthology.orchestra.run.smithi158.stdout: rgw enable usage log = true 2023-10-12T02:13:09.259 INFO:teuthology.orchestra.run.smithi158.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-12T02:13:09.260 INFO:teuthology.orchestra.run.smithi158.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-12T02:13:09.260 INFO:teuthology.orchestra.run.smithi158.stdout: debug ms = 1 2023-10-12T02:13:09.260 INFO:teuthology.orchestra.run.smithi158.stdout:[mon.a] 2023-10-12T02:13:09.260 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-12T02:13:09.261 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-12T02:13:09.334 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-12T02:13:09.335 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:09.335 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-12T02:13:09.350 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:09.350 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-12T02:13:09.404 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi125.front.sepia.ceph.com 2023-10-12T02:13:09.405 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:09.405 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-12T02:13:09.405 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-12T02:13:09.483 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:09.483 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:09.537 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi158.front.sepia.ceph.com 2023-10-12T02:13:09.538 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:09.538 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-12T02:13:09.538 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-12T02:13:09.577 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:09.577 DEBUG:teuthology.orchestra.run.smithi158:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:09.632 INFO:tasks.ceph:Setting up mon nodes... 2023-10-12T02:13:09.632 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-12T02:13:09.633 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-12T02:13:09.759 INFO:teuthology.orchestra.run.smithi158.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-12T02:13:09.762 INFO:tasks.ceph:Setting up mds nodes... 2023-10-12T02:13:09.762 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-12T02:13:09.763 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-12T02:13:09.811 INFO:teuthology.orchestra.run.smithi125.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-12T02:13:09.826 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-12T02:13:09.874 INFO:teuthology.orchestra.run.smithi158.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-12T02:13:09.890 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-12T02:13:09.890 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi125.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi158.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-10-12T02:13:09.890 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-12T02:13:09.917 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-10-12T02:13:09.917 INFO:tasks.ceph:role: osd.0 2023-10-12T02:13:09.918 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-12T02:13:09.918 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-12T02:13:09.990 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-12T02:13:09.991 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-12T02:13:09.991 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-12T02:13:09.991 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-12T02:13:09.991 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-12T02:13:09.992 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2023-10-12T02:13:09.992 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-12T02:13:09.992 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-12T02:13:09.992 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-12T02:13:09.992 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-12T02:13:10.509 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2023-10-12T02:13:10.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2023-10-12T02:13:10.511 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-12T02:13:10.627 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-12T02:13:10.659 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-12T02:13:10.725 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-10-12T02:13:10.726 INFO:tasks.ceph:role: osd.1 2023-10-12T02:13:10.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-12T02:13:10.726 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-12T02:13:10.800 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-12T02:13:10.800 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-12T02:13:10.801 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-12T02:13:10.801 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-12T02:13:10.801 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-12T02:13:10.801 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2023-10-12T02:13:10.802 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-12T02:13:10.802 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-12T02:13:10.802 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-12T02:13:10.802 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-12T02:13:11.281 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2023-10-12T02:13:11.282 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2023-10-12T02:13:11.283 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-12T02:13:11.372 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-12T02:13:11.402 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:11.982 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:11.990 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:11,990 seastar - Reactor backend: linux-aio 2023-10-12T02:13:12.014 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,014 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:12.015 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,014 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:12.015 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,014 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:12.015 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,014 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:12.016 INFO:teuthology.orchestra.run.smithi125.stderr:==79622==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:12.022 INFO:teuthology.orchestra.run.smithi125.stderr:==79622==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc62cf2000; bottom 0x7f04cdf17000; size: 0x00f794ddb000 (1063354478592) 2023-10-12T02:13:12.023 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2023-10-12T02:13:12.023 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:12.057 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,057 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-10-12T02:13:12.057 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,057 [shard 0] osd - get_early_config: set --smp 3 2023-10-12T02:13:12.536 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:12.536 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2023-10-12T02:13:12.537 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2023-10-12T02:13:12.537 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:12.537 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:12.537 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-12T02:13:12.590 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:12.591 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,591 seastar - Reactor backend: linux-aio 2023-10-12T02:13:12.617 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,617 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:12.617 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,617 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:12.618 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,617 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:12.618 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:12,617 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:12.627 INFO:teuthology.orchestra.run.smithi125.stderr:==79620==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:12.633 INFO:teuthology.orchestra.run.smithi125.stderr:==79620==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc62cf2000; bottom 0x7f04cbd01000; size: 0x00f796ff1000 (1063390220288) 2023-10-12T02:13:12.633 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2023-10-12T02:13:12.633 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:15.725 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:15.725 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2023-10-12T02:13:15.725 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2023-10-12T02:13:15.726 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:15.726 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:15.726 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-12T02:13:15.726 INFO:teuthology.orchestra.run.smithi125.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:15.763 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-12T02:13:15.791 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:16.307 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:16.307 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,307 seastar - Reactor backend: linux-aio 2023-10-12T02:13:16.329 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,329 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:16.330 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,329 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:16.330 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,329 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:16.330 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,329 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:16.331 INFO:teuthology.orchestra.run.smithi125.stderr:==80431==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:16.338 INFO:teuthology.orchestra.run.smithi125.stderr:==80431==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff19f50000; bottom 0x7f9102a97000; size: 0x006e174b9000 (472837230592) 2023-10-12T02:13:16.338 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2023-10-12T02:13:16.338 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:16.372 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,372 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-10-12T02:13:16.372 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,372 [shard 0] osd - get_early_config: set --smp 3 2023-10-12T02:13:16.840 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:16.841 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2023-10-12T02:13:16.841 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2023-10-12T02:13:16.841 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:16.841 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:16.842 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-12T02:13:16.892 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:16.893 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,893 seastar - Reactor backend: linux-aio 2023-10-12T02:13:16.911 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,910 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:16.911 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,911 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:16.911 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,911 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:16.918 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2023-10-12 02:13:16,918 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:16.921 INFO:teuthology.orchestra.run.smithi125.stderr:==80429==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:16.926 INFO:teuthology.orchestra.run.smithi125.stderr:==80429==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff19f50000; bottom 0x7f9100880000; size: 0x006e196d0000 (472872976384) 2023-10-12T02:13:16.927 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2023-10-12T02:13:16.927 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:20.263 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:20.263 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2023-10-12T02:13:20.263 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2023-10-12T02:13:20.264 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:20.264 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2023-10-12T02:13:20.264 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-12T02:13:20.264 INFO:teuthology.orchestra.run.smithi125.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:20.315 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-12T02:13:20.341 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-12T02:13:20.373 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-10-12T02:13:20.373 INFO:tasks.ceph:role: osd.2 2023-10-12T02:13:20.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi158.front.sepia.ceph.com 2023-10-12T02:13:20.374 DEBUG:teuthology.orchestra.run.smithi158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-12T02:13:20.443 INFO:teuthology.orchestra.run.smithi158.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-12T02:13:20.443 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-12T02:13:20.443 INFO:teuthology.orchestra.run.smithi158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-12T02:13:20.443 INFO:teuthology.orchestra.run.smithi158.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-12T02:13:20.444 INFO:teuthology.orchestra.run.smithi158.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-12T02:13:20.444 INFO:teuthology.orchestra.run.smithi158.stdout: = sunit=0 swidth=0 blks 2023-10-12T02:13:20.444 INFO:teuthology.orchestra.run.smithi158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-12T02:13:20.444 INFO:teuthology.orchestra.run.smithi158.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-12T02:13:20.444 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-12T02:13:20.445 INFO:teuthology.orchestra.run.smithi158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-12T02:13:20.475 INFO:teuthology.orchestra.run.smithi158.stdout:Discarding blocks...Done. 2023-10-12T02:13:20.477 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi158.front.sepia.ceph.com -o noatime 2023-10-12T02:13:20.478 DEBUG:teuthology.orchestra.run.smithi158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2023-10-12T02:13:20.552 DEBUG:teuthology.orchestra.run.smithi158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-12T02:13:20.625 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-12T02:13:20.691 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-10-12T02:13:20.692 INFO:tasks.ceph:role: osd.3 2023-10-12T02:13:20.692 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi158.front.sepia.ceph.com 2023-10-12T02:13:20.692 DEBUG:teuthology.orchestra.run.smithi158:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-12T02:13:20.758 INFO:teuthology.orchestra.run.smithi158.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-12T02:13:20.758 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-12T02:13:20.759 INFO:teuthology.orchestra.run.smithi158.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-12T02:13:20.759 INFO:teuthology.orchestra.run.smithi158.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-12T02:13:20.759 INFO:teuthology.orchestra.run.smithi158.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-12T02:13:20.760 INFO:teuthology.orchestra.run.smithi158.stdout: = sunit=0 swidth=0 blks 2023-10-12T02:13:20.760 INFO:teuthology.orchestra.run.smithi158.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-12T02:13:20.761 INFO:teuthology.orchestra.run.smithi158.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-12T02:13:20.761 INFO:teuthology.orchestra.run.smithi158.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-12T02:13:20.761 INFO:teuthology.orchestra.run.smithi158.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-12T02:13:20.790 INFO:teuthology.orchestra.run.smithi158.stdout:Discarding blocks...Done. 2023-10-12T02:13:20.792 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi158.front.sepia.ceph.com -o noatime 2023-10-12T02:13:20.793 DEBUG:teuthology.orchestra.run.smithi158:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2023-10-12T02:13:20.865 DEBUG:teuthology.orchestra.run.smithi158:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-12T02:13:20.936 DEBUG:teuthology.orchestra.run.smithi158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:21.469 INFO:teuthology.orchestra.run.smithi158.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:21.470 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:21,470 seastar - Reactor backend: linux-aio 2023-10-12T02:13:21.489 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:21,489 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:21.489 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:21,489 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:21.489 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:21,489 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:21.490 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:21,489 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:21.491 INFO:teuthology.orchestra.run.smithi158.stderr:==79782==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:21.497 INFO:teuthology.orchestra.run.smithi158.stderr:==79782==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd95da1000; bottom 0x7f27c9314000; size: 0x00d5cca8d000 (918261649408) 2023-10-12T02:13:21.497 INFO:teuthology.orchestra.run.smithi158.stderr:False positive error reports may follow 2023-10-12T02:13:21.498 INFO:teuthology.orchestra.run.smithi158.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:21.523 INFO:teuthology.orchestra.run.smithi158.stderr:Reactor stalled for 32 ms on shard 0. Backtrace: 0x45d5d 0x2c844dd6 0x2c846184 0x2c8476d2 0x2c847a56 0x2c847ba0 0x2c847ff6 0x54daf 0xa111c 0xc384c 0xc161c 0x2c68d 0x2857c 0xb6323 0x1e8b31b7 0x1e8b40b3 0x2027edab 0x22ef6f35 0x22d61515 0x22bb9b84 0x22be75d7 0x24ba4ba1 0x24baa706 0x24bad054 0x24bae22d 0x22774737 0x227c88fe 0x227e84fb 0x2c7e70f1 0x2c884f0c 0x2caa8d0e 0x2caaab59 0x2c5030d4 0x2c504fd5 0x1eed4f0c 0x1eedf079 0x1ed59731 0x3feaf 0x3ff5f 0x1e812a04 2023-10-12T02:13:21.524 INFO:teuthology.orchestra.run.smithi158.stderr:kernel callstack: 2023-10-12T02:13:21.532 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:21,532 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-10-12T02:13:21.532 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:21,532 [shard 0] osd - get_early_config: set --smp 3 2023-10-12T02:13:22.037 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:22.037 INFO:teuthology.orchestra.run.smithi158.stderr:Suppressions used: 2023-10-12T02:13:22.038 INFO:teuthology.orchestra.run.smithi158.stderr: count bytes template 2023-10-12T02:13:22.038 INFO:teuthology.orchestra.run.smithi158.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:22.038 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:22.038 INFO:teuthology.orchestra.run.smithi158.stderr: 2023-10-12T02:13:22.086 INFO:teuthology.orchestra.run.smithi158.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:22.087 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:22,086 seastar - Reactor backend: linux-aio 2023-10-12T02:13:22.104 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:22,104 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:22.105 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:22,104 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:22.105 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:22,104 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:22.112 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:22,112 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:22.115 INFO:teuthology.orchestra.run.smithi158.stderr:==79773==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:22.120 INFO:teuthology.orchestra.run.smithi158.stderr:==79773==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd95da1000; bottom 0x7f27c70fd000; size: 0x00d5ceca4000 (918297395200) 2023-10-12T02:13:22.121 INFO:teuthology.orchestra.run.smithi158.stderr:False positive error reports may follow 2023-10-12T02:13:22.121 INFO:teuthology.orchestra.run.smithi158.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:22.202 INFO:teuthology.orchestra.run.smithi158.stderr:Reactor stalled for 33 ms on shard 0. Backtrace: 0x45d5d 0x2c844dd6 0x2c846184 0x2c8476d2 0x2c847a56 0x2c847ba0 0x2c847ff6 0x54daf 0xce1ca 0x588c4 0xdb9f8 0x2640d0dc 0x2640f725 0x26411536 0x26413d9b 0x2624394f 0x1ee15204 0x1ed896f5 0x1ed8d661 0x2cbc8a39 2023-10-12T02:13:22.202 INFO:teuthology.orchestra.run.smithi158.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffade5d817 0xffffffffade59167 0xffffffffae0000aa 2023-10-12T02:13:25.187 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:25.187 INFO:teuthology.orchestra.run.smithi158.stderr:Suppressions used: 2023-10-12T02:13:25.188 INFO:teuthology.orchestra.run.smithi158.stderr: count bytes template 2023-10-12T02:13:25.188 INFO:teuthology.orchestra.run.smithi158.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:25.188 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:25.188 INFO:teuthology.orchestra.run.smithi158.stderr: 2023-10-12T02:13:25.189 INFO:teuthology.orchestra.run.smithi158.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:25.226 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-12T02:13:25.254 DEBUG:teuthology.orchestra.run.smithi158:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:25.761 INFO:teuthology.orchestra.run.smithi158.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:25.762 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:25,762 seastar - Reactor backend: linux-aio 2023-10-12T02:13:25.782 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:25,781 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:25.782 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:25,781 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:25.782 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:25,782 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:25.782 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:25,782 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:25.784 INFO:teuthology.orchestra.run.smithi158.stderr:==80591==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:25.790 INFO:teuthology.orchestra.run.smithi158.stderr:==80591==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe5d82f000; bottom 0x7fe5a8b71000; size: 0x0018b4cbe000 (106112475136) 2023-10-12T02:13:25.790 INFO:teuthology.orchestra.run.smithi158.stderr:False positive error reports may follow 2023-10-12T02:13:25.790 INFO:teuthology.orchestra.run.smithi158.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:25.824 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:25,824 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-10-12T02:13:25.824 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:25,824 [shard 0] osd - get_early_config: set --smp 3 2023-10-12T02:13:26.303 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:26.303 INFO:teuthology.orchestra.run.smithi158.stderr:Suppressions used: 2023-10-12T02:13:26.303 INFO:teuthology.orchestra.run.smithi158.stderr: count bytes template 2023-10-12T02:13:26.303 INFO:teuthology.orchestra.run.smithi158.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:26.304 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:26.304 INFO:teuthology.orchestra.run.smithi158.stderr: 2023-10-12T02:13:26.361 INFO:teuthology.orchestra.run.smithi158.stderr:WARNING: debug mode. Not for benchmarking or production 2023-10-12T02:13:26.362 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:26,362 seastar - Reactor backend: linux-aio 2023-10-12T02:13:26.379 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:26,379 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-10-12T02:13:26.380 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:26,379 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-10-12T02:13:26.380 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:26,379 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-10-12T02:13:26.387 INFO:teuthology.orchestra.run.smithi158.stderr:INFO 2023-10-12 02:13:26,387 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-10-12T02:13:26.389 INFO:teuthology.orchestra.run.smithi158.stderr:==80589==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-10-12T02:13:26.396 INFO:teuthology.orchestra.run.smithi158.stderr:==80589==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe5d82f000; bottom 0x7fe5a695a000; size: 0x0018b6ed5000 (106148220928) 2023-10-12T02:13:26.396 INFO:teuthology.orchestra.run.smithi158.stderr:False positive error reports may follow 2023-10-12T02:13:26.396 INFO:teuthology.orchestra.run.smithi158.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-10-12T02:13:26.476 INFO:teuthology.orchestra.run.smithi158.stderr:Reactor stalled for 33 ms on shard 0. Backtrace: 0x45d5d 0x2c844dd6 0x2c846184 0x2c8476d2 0x2c847a56 0x2c847ba0 0x2c847ff6 0x54daf 0xce1ca 0x588c4 0xdb9f8 0x2640d0dc 0x2640f725 0x26411536 0x26413d9b 0x2624394f 0x1ee15204 0x1ed896f5 0x1ed8d661 0x2cbc8a39 2023-10-12T02:13:26.477 INFO:teuthology.orchestra.run.smithi158.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffad34fa43 0xffffffffad34fa8a 0xffffffffade5918c 0xffffffffae0000aa 2023-10-12T02:13:29.453 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:29.454 INFO:teuthology.orchestra.run.smithi158.stderr:Suppressions used: 2023-10-12T02:13:29.454 INFO:teuthology.orchestra.run.smithi158.stderr: count bytes template 2023-10-12T02:13:29.454 INFO:teuthology.orchestra.run.smithi158.stderr: 1 8 MallocExtension::Register 2023-10-12T02:13:29.454 INFO:teuthology.orchestra.run.smithi158.stderr:----------------------------------------------------- 2023-10-12T02:13:29.454 INFO:teuthology.orchestra.run.smithi158.stderr: 2023-10-12T02:13:29.455 INFO:teuthology.orchestra.run.smithi158.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:13:29.504 DEBUG:teuthology.orchestra.run.smithi158:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-12T02:13:29.532 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-12T02:13:29.532 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:29.533 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-12T02:13:29.565 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:29.565 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-12T02:13:29.634 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:29.635 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-12T02:13:29.662 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:29.662 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-12T02:13:29.729 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:29.729 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-12T02:13:29.795 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:29.796 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-12T02:13:29.813 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:29.814 DEBUG:teuthology.orchestra.run.smithi158:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-12T02:13:29.850 INFO:tasks.ceph:Adding keys to all mons... 2023-10-12T02:13:29.851 DEBUG:teuthology.orchestra.run.smithi125:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-12T02:13:29.882 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.0] 2023-10-12T02:13:29.882 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQA4VidltXqaKBAAi7V67afcMn6JQA1h59QuyA== 2023-10-12T02:13:29.883 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.1] 2023-10-12T02:13:29.883 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQA8Vidl1oUmOhAA/1OR73URFIQKM/6zU5SGEA== 2023-10-12T02:13:29.883 INFO:teuthology.orchestra.run.smithi125.stdout:[mgr.x] 2023-10-12T02:13:29.883 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQA1Vidl24BELRAAHeCk2nh9Wq/vcGEekV5FxQ== 2023-10-12T02:13:29.883 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.2] 2023-10-12T02:13:29.884 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBCVidlcOUPChAApvOcs3c0HT+Q88nXULAg4A== 2023-10-12T02:13:29.884 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.3] 2023-10-12T02:13:29.884 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBGVidlvk11GhAAk1uMOs6qO4Mj8+vWvgdZDA== 2023-10-12T02:13:29.884 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-12T02:13:29.884 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQA1VidlslJZMBAAVbBXLzXhfZrjOzVfd8wz7g== 2023-10-12T02:13:29.885 INFO:teuthology.orchestra.run.smithi125.stdout:[client.1] 2023-10-12T02:13:29.885 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQA1Vidl6GEhNBAAWuYfkdAm/5XzXQLMZzmI1Q== 2023-10-12T02:13:29.886 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-12T02:13:29.990 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-12T02:13:30.049 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-12T02:13:30.108 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-12T02:13:30.166 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-12T02:13:30.224 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-12T02:13:30.286 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-12T02:13:30.349 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-12T02:13:30.349 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-12T02:13:30.376 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-12T02:13:30.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.473+0000 7f27c022e980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.473+0000 7f27c022e980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.490 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-12T02:13:30.517 DEBUG:teuthology.orchestra.run.smithi125:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-12T02:13:30.571 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-12T02:13:30.572 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-12T02:13:30.572 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-12T02:13:30.613 INFO:tasks.ceph.mon.a:Started 2023-10-12T02:13:30.613 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-12T02:13:30.614 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-12T02:13:30.614 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-12T02:13:30.617 INFO:tasks.ceph.mgr.x:Started 2023-10-12T02:13:30.617 INFO:tasks.ceph:Setting crush tunables to default 2023-10-12T02:13:30.617 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd crush tunables default 2023-10-12T02:13:30.691 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:13:30.690+0000 7f75cd82b980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.692 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:13:30.690+0000 7f75cd82b980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.692 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:13:30.690+0000 7f75cd82b980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.709 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:30.707+0000 7f774c228f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.717+0000 7fbb91a0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.724+0000 7fbb91a0f640 1 Processor -- start 2023-10-12T02:13:30.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.724+0000 7fbb91a0f640 1 -- start start 2023-10-12T02:13:30.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.724+0000 7fbb91a0f640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 0x7fbb8c133440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:30.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.724+0000 7fbb91a0f640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c06c210 con 0x7fbb8c133980 2023-10-12T02:13:30.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.724+0000 7fbb91a0f640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c06cdb0 con 0x7fbb8c06d140 2023-10-12T02:13:30.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb8affd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 0x7fbb8c133440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:30.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb83fff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fbb8c133980 0x7fbb8c12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:44524/0 (socket says 172.21.15.125:44524) 2023-10-12T02:13:30.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb8affd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 0x7fbb8c133440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54272/0 (socket says 172.21.15.125:54272) 2023-10-12T02:13:30.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb83fff640 1 -- 172.21.15.125:0/1725837701 learned_addr learned my addr 172.21.15.125:0/1725837701 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:30.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb8affd640 1 -- 172.21.15.125:0/1725837701 >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 msgr2=0x7fbb8c133440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-12T02:13:30.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb8affd640 1 -- 172.21.15.125:0/1725837701 >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 msgr2=0x7fbb8c133440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-12T02:13:30.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb8affd640 1 --2- 172.21.15.125:0/1725837701 >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 0x7fbb8c133440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-12T02:13:30.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.725+0000 7fbb8affd640 1 --2- 172.21.15.125:0/1725837701 >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 0x7fbb8c133440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T02:13:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291795071 0 0) 0x7fbb8c06c210 con 0x7fbb8c133980 2023-10-12T02:13:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb780034f0 con 0x7fbb8c133980 2023-10-12T02:13:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 <== mon.0 v1:172.21.15.125:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506044429 0 0) 0x7fbb780034f0 con 0x7fbb8c133980 2023-10-12T02:13:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 msgr2=0x7fbb8c133440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T02:13:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 --2- 172.21.15.125:0/1725837701 >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 0x7fbb8c133440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 --> v1:172.21.15.125:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c06d550 con 0x7fbb8c133980 2023-10-12T02:13:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 <== mon.0 v1:172.21.15.125:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 4071500210 0 0) 0x7fbb740015a0 con 0x7fbb8c133980 2023-10-12T02:13:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 >> v1:172.21.15.125:6789/0 conn(0x7fbb8c133980 legacy=0x7fbb8c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 --2- 172.21.15.125:0/1725837701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.751+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c06cdb0 con 0x7fbb78003940 2023-10-12T02:13:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.752+0000 7fbb8affd640 1 --2- 172.21.15.125:0/1725837701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.752+0000 7fbb8affd640 1 -- 172.21.15.125:0/1725837701 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780034f0 con 0x7fbb78003940 2023-10-12T02:13:30.756 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:30.754+0000 7f774c228f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.757 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:30.755+0000 7f774c228f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.754+0000 7fbb8affd640 1 --2- 172.21.15.125:0/1725837701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb78003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e434938edec18a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:30.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.754+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbb7c00b5b0 con 0x7fbb78003940 2023-10-12T02:13:30.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.754+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbb7c00baf0 con 0x7fbb78003940 2023-10-12T02:13:30.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.754+0000 7fbb8a7fc640 1 -- 172.21.15.125:0/1725837701 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbb7c00bd90 con 0x7fbb78003940 2023-10-12T02:13:30.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 -- 172.21.15.125:0/1725837701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 msgr2=0x7fbb78003d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:30.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 --2- 172.21.15.125:0/1725837701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb78003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 -- 172.21.15.125:0/1725837701 shutdown_connections 2023-10-12T02:13:30.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 --2- 172.21.15.125:0/1725837701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb78003d90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 --2- 172.21.15.125:0/1725837701 >> v2:172.21.15.125:3300/0 conn(0x7fbb8c06d140 0x7fbb8c133440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 -- 172.21.15.125:0/1725837701 >> 172.21.15.125:0/1725837701 conn(0x7fbb8c0a2d60 msgr2=0x7fbb8c0a3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:30.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 -- 172.21.15.125:0/1725837701 shutdown_connections 2023-10-12T02:13:30.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 1 -- 172.21.15.125:0/1725837701 wait complete. 2023-10-12T02:13:30.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.755+0000 7fbb91a0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.756+0000 7fbb91a0f640 1 Processor -- start 2023-10-12T02:13:30.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.756+0000 7fbb91a0f640 1 -- start start 2023-10-12T02:13:30.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.756+0000 7fbb91a0f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb8c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:30.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.756+0000 7fbb91a0f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c067db0 con 0x7fbb78003940 2023-10-12T02:13:30.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb8c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:30.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb8c06ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54298/0 (socket says 172.21.15.125:54298) 2023-10-12T02:13:30.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb8affd640 1 -- 172.21.15.125:0/2834662362 learned_addr learned my addr 172.21.15.125:0/2834662362 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:30.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb8affd640 1 -- 172.21.15.125:0/2834662362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c06f980 con 0x7fbb78003940 2023-10-12T02:13:30.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb8affd640 1 --2- 172.21.15.125:0/2834662362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb8c06ff20 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:30.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb88ff9640 1 -- 172.21.15.125:0/2834662362 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbb7c003710 con 0x7fbb78003940 2023-10-12T02:13:30.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb88ff9640 1 -- 172.21.15.125:0/2834662362 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbb7c003870 con 0x7fbb78003940 2023-10-12T02:13:30.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c11bee0 con 0x7fbb78003940 2023-10-12T02:13:30.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb88ff9640 1 -- 172.21.15.125:0/2834662362 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbb7c003b10 con 0x7fbb78003940 2023-10-12T02:13:30.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c11c830 con 0x7fbb78003940 2023-10-12T02:13:30.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb88ff9640 1 -- 172.21.15.125:0/2834662362 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7fbb7c016430 con 0x7fbb78003940 2023-10-12T02:13:30.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.757+0000 7fbb88ff9640 1 -- 172.21.15.125:0/2834662362 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (crc 0 0 0) 0x7fbb7c0157b0 con 0x7fbb78003940 2023-10-12T02:13:30.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.758+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c071120 con 0x7fbb78003940 2023-10-12T02:13:30.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.760+0000 7fbb88ff9640 1 -- 172.21.15.125:0/2834662362 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7fbb7c015970 con 0x7fbb78003940 2023-10-12T02:13:30.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.822+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbb8c11c0d0 con 0x7fbb78003940 2023-10-12T02:13:30.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.823+0000 7fbb88ff9640 1 -- 172.21.15.125:0/2834662362 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (crc 0 0 0) 0x7fbb7c015b10 con 0x7fbb78003940 2023-10-12T02:13:30.824 INFO:teuthology.orchestra.run.smithi125.stderr:adjusted tunables profile to default 2023-10-12T02:13:30.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.824+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 msgr2=0x7fbb8c06ff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:30.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.824+0000 7fbb91a0f640 1 --2- 172.21.15.125:0/2834662362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb8c06ff20 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.824+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 shutdown_connections 2023-10-12T02:13:30.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.824+0000 7fbb91a0f640 1 --2- 172.21.15.125:0/2834662362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb78003940 0x7fbb8c06ff20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.824+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 >> 172.21.15.125:0/2834662362 conn(0x7fbb8c0a2d60 msgr2=0x7fbb8c1bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:30.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.824+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 shutdown_connections 2023-10-12T02:13:30.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.824+0000 7fbb91a0f640 1 -- 172.21.15.125:0/2834662362 wait complete. 2023-10-12T02:13:30.835 INFO:tasks.ceph:check_enable_crimson: True 2023-10-12T02:13:30.835 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2023-10-12T02:13:30.836 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2023-10-12T02:13:30.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.934+0000 7f1e8326d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.939+0000 7f1e8326d640 1 Processor -- start 2023-10-12T02:13:30.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.939+0000 7f1e8326d640 1 -- start start 2023-10-12T02:13:30.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e8326d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f1e7c12d010 0x7f1e7c134d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:30.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e8326d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c12c540 con 0x7f1e7c06d800 2023-10-12T02:13:30.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e8326d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c06e990 con 0x7f1e7c12d010 2023-10-12T02:13:30.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e80808640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f1e7c12d010 0x7f1e7c134d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:30.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e81009640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f1e7c06d800 0x7f1e7c12c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:44532/0 (socket says 172.21.15.125:44532) 2023-10-12T02:13:30.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e81009640 1 -- 172.21.15.125:0/2802110890 learned_addr learned my addr 172.21.15.125:0/2802110890 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:30.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e80808640 1 -- 172.21.15.125:0/2802110890 >> v1:172.21.15.125:6789/0 conn(0x7f1e7c06d800 legacy=0x7f1e7c12c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:30.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e80808640 1 -- 172.21.15.125:0/2802110890 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c12d420 con 0x7f1e7c12d010 2023-10-12T02:13:30.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e73fff640 1 -- 172.21.15.125:0/2802110890 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587422157 0 0) 0x7f1e7c12c540 con 0x7f1e7c06d800 2023-10-12T02:13:30.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e80808640 1 --2- 172.21.15.125:0/2802110890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c12d010 0x7f1e7c134d80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d158e60d0535d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:30.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e73fff640 1 -- 172.21.15.125:0/2802110890 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1e6c009ce0 con 0x7f1e7c12d010 2023-10-12T02:13:30.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e73fff640 1 -- 172.21.15.125:0/2802110890 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1e6c009e40 con 0x7f1e7c12d010 2023-10-12T02:13:30.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.940+0000 7f1e73fff640 1 -- 172.21.15.125:0/2802110890 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1e6c00e360 con 0x7f1e7c12d010 2023-10-12T02:13:30.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 1 -- 172.21.15.125:0/2802110890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c12d010 msgr2=0x7f1e7c134d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:30.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 1 --2- 172.21.15.125:0/2802110890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c12d010 0x7f1e7c134d80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 1 -- 172.21.15.125:0/2802110890 shutdown_connections 2023-10-12T02:13:30.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 1 --2- 172.21.15.125:0/2802110890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c12d010 0x7f1e7c134d80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:30.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 1 -- 172.21.15.125:0/2802110890 >> 172.21.15.125:0/2802110890 conn(0x7f1e7c09f840 msgr2=0x7f1e7c09fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:30.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 1 -- 172.21.15.125:0/2802110890 shutdown_connections 2023-10-12T02:13:30.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 1 -- 172.21.15.125:0/2802110890 wait complete. 2023-10-12T02:13:30.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.941+0000 7f1e8326d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:30.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.942+0000 7f1e8326d640 1 Processor -- start 2023-10-12T02:13:30.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.942+0000 7f1e8326d640 1 -- start start 2023-10-12T02:13:30.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e8326d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c06d800 0x7f1e7c070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:30.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e8326d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c1198f0 con 0x7f1e7c06d800 2023-10-12T02:13:30.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e81009640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c06d800 0x7f1e7c070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:30.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e81009640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c06d800 0x7f1e7c070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54302/0 (socket says 172.21.15.125:54302) 2023-10-12T02:13:30.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e81009640 1 -- 172.21.15.125:0/2762065129 learned_addr learned my addr 172.21.15.125:0/2762065129 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:30.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e81009640 1 -- 172.21.15.125:0/2762065129 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0708e0 con 0x7f1e7c06d800 2023-10-12T02:13:30.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e81009640 1 --2- 172.21.15.125:0/2762065129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c06d800 0x7f1e7c070ea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:30.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e71ffb640 1 -- 172.21.15.125:0/2762065129 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1e78004af0 con 0x7f1e7c06d800 2023-10-12T02:13:30.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e71ffb640 1 -- 172.21.15.125:0/2762065129 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1e78004c50 con 0x7f1e7c06d800 2023-10-12T02:13:30.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c0705d0 con 0x7f1e7c06d800 2023-10-12T02:13:30.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.943+0000 7f1e71ffb640 1 -- 172.21.15.125:0/2762065129 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1e78007120 con 0x7f1e7c06d800 2023-10-12T02:13:30.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.944+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c118c00 con 0x7f1e7c06d800 2023-10-12T02:13:30.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.944+0000 7f1e71ffb640 1 -- 172.21.15.125:0/2762065129 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f1e7800c4e0 con 0x7f1e7c06d800 2023-10-12T02:13:30.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.944+0000 7f1e71ffb640 1 -- 172.21.15.125:0/2762065129 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (crc 0 0 0) 0x7f1e7800e640 con 0x7f1e7c06d800 2023-10-12T02:13:30.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.944+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c129f00 con 0x7f1e7c06d800 2023-10-12T02:13:30.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:30.946+0000 7f1e71ffb640 1 -- 172.21.15.125:0/2762065129 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7f1e7800ca10 con 0x7f1e7c06d800 2023-10-12T02:13:31.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.018+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1e7c061d90 con 0x7f1e7c06d800 2023-10-12T02:13:31.019 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:13:31.018+0000 7f75c4bf1640 -1 WARNING: experimental feature 'crimson' is enabled 2023-10-12T02:13:31.020 INFO:tasks.ceph.mon.a.smithi125.stderr:Please be aware that this feature is experimental, untested, 2023-10-12T02:13:31.020 INFO:tasks.ceph.mon.a.smithi125.stderr:unsupported, and may result in data corruption, data loss, 2023-10-12T02:13:31.020 INFO:tasks.ceph.mon.a.smithi125.stderr:and/or irreparable damage to your cluster. Do not use 2023-10-12T02:13:31.020 INFO:tasks.ceph.mon.a.smithi125.stderr:feature with important data. 2023-10-12T02:13:31.020 INFO:tasks.ceph.mon.a.smithi125.stderr: 2023-10-12T02:13:31.208 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:31.206+0000 7f774c228f00 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-12T02:13:31.258 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:31.256+0000 7f774c228f00 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-12T02:13:31.818 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:31.817+0000 7f774c228f00 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-12T02:13:31.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.824+0000 7f1e71ffb640 1 -- 172.21.15.125:0/2762065129 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) v1 ==== 99+0+0 (crc 0 0 0) 0x7f1e7800cc80 con 0x7f1e7c06d800 2023-10-12T02:13:31.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.825+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c06d800 msgr2=0x7f1e7c070ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:31.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.825+0000 7f1e8326d640 1 --2- 172.21.15.125:0/2762065129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c06d800 0x7f1e7c070ea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.825+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 shutdown_connections 2023-10-12T02:13:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.825+0000 7f1e8326d640 1 --2- 172.21.15.125:0/2762065129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e7c06d800 0x7f1e7c070ea0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.825+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 >> 172.21.15.125:0/2762065129 conn(0x7f1e7c09f840 msgr2=0x7f1e7c12b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.825+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 shutdown_connections 2023-10-12T02:13:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:31.825+0000 7f1e8326d640 1 -- 172.21.15.125:0/2762065129 wait complete. 2023-10-12T02:13:31.837 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-12T02:13:31.838 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:31.838 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-12T02:13:31.866 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:13:31.866 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-12T02:13:31.870 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:31.869+0000 7f774c228f00 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-12T02:13:31.920 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:31.918+0000 7f774c228f00 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-12T02:13:31.933 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:31.933 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-12T02:13:31.961 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2023-10-12T02:13:31.961 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-12T02:13:31.983 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:31.981+0000 7f774c228f00 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-12T02:13:32.009 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5 0 2023-10-12T02:13:32.028 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.026+0000 7f774c228f00 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-12T02:13:32.134 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.127+0000 7f595822d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.134 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f595822d640 1 Processor -- start 2023-10-12T02:13:32.135 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f595822d640 1 -- start start 2023-10-12T02:13:32.137 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f595822d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f595011a990 0x7f595011cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.137 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f595822d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595011d380 con 0x7f5950111d80 2023-10-12T02:13:32.137 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f595822d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595011e520 con 0x7f595011a990 2023-10-12T02:13:32.137 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f59557c8640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f595011a990 0x7f595011cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.138 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f5955fc9640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f5950111d80 0x7f5950112190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.158:33004/0 (socket says 172.21.15.158:33004) 2023-10-12T02:13:32.138 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.133+0000 7f5955fc9640 1 -- 172.21.15.158:0/3012157965 learned_addr learned my addr 172.21.15.158:0/3012157965 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-10-12T02:13:32.138 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.134+0000 7f59557c8640 1 -- 172.21.15.158:0/3012157965 >> v1:172.21.15.125:6789/0 conn(0x7f5950111d80 legacy=0x7f5950112190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.138 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.134+0000 7f59557c8640 1 -- 172.21.15.158:0/3012157965 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595011e660 con 0x7f595011a990 2023-10-12T02:13:32.139 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.134+0000 7f59557c8640 1 --2- 172.21.15.158:0/3012157965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f595011a990 0x7f595011cdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679760fbe864af59 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.139 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.134+0000 7f5954fc7640 1 -- 172.21.15.158:0/3012157965 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5940009ce0 con 0x7f595011a990 2023-10-12T02:13:32.139 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.134+0000 7f5954fc7640 1 -- 172.21.15.158:0/3012157965 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5940009e40 con 0x7f595011a990 2023-10-12T02:13:32.139 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.134+0000 7f5954fc7640 1 -- 172.21.15.158:0/3012157965 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f594000e360 con 0x7f595011a990 2023-10-12T02:13:32.139 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 1 -- 172.21.15.158:0/3012157965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f595011a990 msgr2=0x7f595011cdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.140 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 1 --2- 172.21.15.158:0/3012157965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f595011a990 0x7f595011cdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.140 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 1 -- 172.21.15.158:0/3012157965 shutdown_connections 2023-10-12T02:13:32.140 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 1 --2- 172.21.15.158:0/3012157965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f595011a990 0x7f595011cdb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.140 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 1 -- 172.21.15.158:0/3012157965 >> 172.21.15.158:0/3012157965 conn(0x7f595006d7b0 msgr2=0x7f595010fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.140 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 1 -- 172.21.15.158:0/3012157965 shutdown_connections 2023-10-12T02:13:32.141 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 1 -- 172.21.15.158:0/3012157965 wait complete. 2023-10-12T02:13:32.141 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f595822d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.141 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.135+0000 7f774c228f00 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-12T02:13:32.143 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f595822d640 1 Processor -- start 2023-10-12T02:13:32.143 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f595822d640 1 -- start start 2023-10-12T02:13:32.144 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f595822d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950111d80 0x7f59501ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.144 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f595822d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59501ac8a0 con 0x7f5950111d80 2023-10-12T02:13:32.144 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f5955fc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950111d80 0x7f59501ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.144 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f5955fc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950111d80 0x7f59501ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.158:54156/0 (socket says 172.21.15.158:54156) 2023-10-12T02:13:32.144 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f5955fc9640 1 -- 172.21.15.158:0/814636450 learned_addr learned my addr 172.21.15.158:0/814636450 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-10-12T02:13:32.145 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f5955fc9640 1 -- 172.21.15.158:0/814636450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59501ac9e0 con 0x7f5950111d80 2023-10-12T02:13:32.145 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.136+0000 7f5955fc9640 1 --2- 172.21.15.158:0/814636450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950111d80 0x7f59501ac2d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.145 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f59467fc640 1 -- 172.21.15.158:0/814636450 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f594c0059d0 con 0x7f5950111d80 2023-10-12T02:13:32.145 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f59467fc640 1 -- 172.21.15.158:0/814636450 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f594c005b30 con 0x7f5950111d80 2023-10-12T02:13:32.145 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59501acc30 con 0x7f5950111d80 2023-10-12T02:13:32.146 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f59467fc640 1 -- 172.21.15.158:0/814636450 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f594c005dd0 con 0x7f5950111d80 2023-10-12T02:13:32.146 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59501a6640 con 0x7f5950111d80 2023-10-12T02:13:32.146 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f59467fc640 1 -- 172.21.15.158:0/814636450 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f594c0073b0 con 0x7f5950111d80 2023-10-12T02:13:32.146 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f59467fc640 1 -- 172.21.15.158:0/814636450 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 747+0+0 (crc 0 0 0) 0x7f594c00f620 con 0x7f5950111d80 2023-10-12T02:13:32.147 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.137+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950061d90 con 0x7f5950111d80 2023-10-12T02:13:32.147 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.140+0000 7f59467fc640 1 -- 172.21.15.158:0/814636450 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7f594c00c770 con 0x7f5950111d80 2023-10-12T02:13:32.210 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.208+0000 7f774c228f00 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-12T02:13:32.213 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.211+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5", "id": 0} v 0) v1 -- 0x7f59501a7100 con 0x7f5950111d80 2023-10-12T02:13:32.214 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.212+0000 7f59467fc640 1 -- 172.21.15.158:0/814636450 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5", "id": 0}]=0 v4) v1 ==== 112+0+2 (crc 0 0 0) 0x7f594c00c910 con 0x7f5950111d80 2023-10-12T02:13:32.215 INFO:teuthology.orchestra.run.smithi158.stdout:0 2023-10-12T02:13:32.216 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.213+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950111d80 msgr2=0x7f59501ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.216 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.213+0000 7f595822d640 1 --2- 172.21.15.158:0/814636450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950111d80 0x7f59501ac2d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.217 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.213+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 shutdown_connections 2023-10-12T02:13:32.217 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.213+0000 7f595822d640 1 --2- 172.21.15.158:0/814636450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950111d80 0x7f59501ac2d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.217 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.213+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 >> 172.21.15.158:0/814636450 conn(0x7f595006d7b0 msgr2=0x7f595011afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.218 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.213+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 shutdown_connections 2023-10-12T02:13:32.218 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.213+0000 7f595822d640 1 -- 172.21.15.158:0/814636450 wait complete. 2023-10-12T02:13:32.225 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new e052e92d-fdbb-4870-b947-c74914b8c0a2 1 2023-10-12T02:13:32.260 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.259+0000 7f774c228f00 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-12T02:13:32.317 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.315+0000 7f774c228f00 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-12T02:13:32.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.325+0000 7f6ab046d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.330+0000 7f6ab046d640 1 Processor -- start 2023-10-12T02:13:32.332 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.330+0000 7f6ab046d640 1 -- start start 2023-10-12T02:13:32.334 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.331+0000 7f6ab046d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f6aa811a9b0 0x7f6aa811cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.334 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.331+0000 7f6ab046d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa811d3a0 con 0x7f6aa8111d00 2023-10-12T02:13:32.334 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.331+0000 7f6ab046d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa811e540 con 0x7f6aa811a9b0 2023-10-12T02:13:32.334 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.331+0000 7f6aada08640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f6aa811a9b0 0x7f6aa811cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.335 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.331+0000 7f6aae209640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f6aa8111d00 0x7f6aa8112110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.158:33020/0 (socket says 172.21.15.158:33020) 2023-10-12T02:13:32.335 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.331+0000 7f6aae209640 1 -- 172.21.15.158:0/2647958206 learned_addr learned my addr 172.21.15.158:0/2647958206 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-10-12T02:13:32.335 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.332+0000 7f6aada08640 1 -- 172.21.15.158:0/2647958206 >> v1:172.21.15.125:6789/0 conn(0x7f6aa8111d00 legacy=0x7f6aa8112110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.335 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.332+0000 7f6aad207640 1 -- 172.21.15.158:0/2647958206 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708533290 0 0) 0x7f6aa811d3a0 con 0x7f6aa8111d00 2023-10-12T02:13:32.335 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.332+0000 7f6aada08640 1 -- 172.21.15.158:0/2647958206 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa811e680 con 0x7f6aa811a9b0 2023-10-12T02:13:32.336 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.332+0000 7f6aada08640 1 --2- 172.21.15.158:0/2647958206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa811a9b0 0x7f6aa811cdd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47f3152a3042f45 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.336 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.332+0000 7f6aad207640 1 -- 172.21.15.158:0/2647958206 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6a98009ce0 con 0x7f6aa811a9b0 2023-10-12T02:13:32.336 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.332+0000 7f6aad207640 1 -- 172.21.15.158:0/2647958206 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6a98009e40 con 0x7f6aa811a9b0 2023-10-12T02:13:32.336 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.332+0000 7f6aad207640 1 -- 172.21.15.158:0/2647958206 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6a9800e360 con 0x7f6aa811a9b0 2023-10-12T02:13:32.339 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 1 -- 172.21.15.158:0/2647958206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa811a9b0 msgr2=0x7f6aa811cdd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.339 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 1 --2- 172.21.15.158:0/2647958206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa811a9b0 0x7f6aa811cdd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.339 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 1 -- 172.21.15.158:0/2647958206 shutdown_connections 2023-10-12T02:13:32.339 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 1 --2- 172.21.15.158:0/2647958206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa811a9b0 0x7f6aa811cdd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.340 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 1 -- 172.21.15.158:0/2647958206 >> 172.21.15.158:0/2647958206 conn(0x7f6aa806d820 msgr2=0x7f6aa810fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.340 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 1 -- 172.21.15.158:0/2647958206 shutdown_connections 2023-10-12T02:13:32.340 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 1 -- 172.21.15.158:0/2647958206 wait complete. 2023-10-12T02:13:32.340 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.333+0000 7f6ab046d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.340 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6ab046d640 1 Processor -- start 2023-10-12T02:13:32.341 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6ab046d640 1 -- start start 2023-10-12T02:13:32.341 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6ab046d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa8111d00 0x7f6aa81ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.341 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6ab046d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa81ac720 con 0x7f6aa8111d00 2023-10-12T02:13:32.341 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6aae209640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa8111d00 0x7f6aa81ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.341 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6aae209640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa8111d00 0x7f6aa81ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.158:54176/0 (socket says 172.21.15.158:54176) 2023-10-12T02:13:32.342 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6aae209640 1 -- 172.21.15.158:0/3919453285 learned_addr learned my addr 172.21.15.158:0/3919453285 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-10-12T02:13:32.342 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.334+0000 7f6aae209640 1 -- 172.21.15.158:0/3919453285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa81ac860 con 0x7f6aa8111d00 2023-10-12T02:13:32.342 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6aae209640 1 --2- 172.21.15.158:0/3919453285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa8111d00 0x7f6aa81ac150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.342 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6a9effd640 1 -- 172.21.15.158:0/3919453285 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6aa4004bd0 con 0x7f6aa8111d00 2023-10-12T02:13:32.342 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6a9effd640 1 -- 172.21.15.158:0/3919453285 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6aa4004d30 con 0x7f6aa8111d00 2023-10-12T02:13:32.343 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa81acab0 con 0x7f6aa8111d00 2023-10-12T02:13:32.343 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6a9effd640 1 -- 172.21.15.158:0/3919453285 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6aa4007120 con 0x7f6aa8111d00 2023-10-12T02:13:32.343 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa81a64c0 con 0x7f6aa8111d00 2023-10-12T02:13:32.343 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6a9effd640 1 -- 172.21.15.158:0/3919453285 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f6aa400c580 con 0x7f6aa8111d00 2023-10-12T02:13:32.343 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.335+0000 7f6a9effd640 1 -- 172.21.15.158:0/3919453285 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 862+0+0 (crc 0 0 0) 0x7f6aa400f780 con 0x7f6aa8111d00 2023-10-12T02:13:32.344 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.336+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa81acca0 con 0x7f6aa8111d00 2023-10-12T02:13:32.344 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.338+0000 7f6a9effd640 1 -- 172.21.15.158:0/3919453285 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7f6aa400ca10 con 0x7f6aa8111d00 2023-10-12T02:13:32.367 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.365+0000 7f774c228f00 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-12T02:13:32.409 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.408+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e052e92d-fdbb-4870-b947-c74914b8c0a2", "id": 1} v 0) v1 -- 0x7f6aa811a9b0 con 0x7f6aa8111d00 2023-10-12T02:13:32.411 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.409+0000 7f6a9effd640 1 -- 172.21.15.158:0/3919453285 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e052e92d-fdbb-4870-b947-c74914b8c0a2", "id": 1}]=0 v5) v1 ==== 112+0+2 (crc 0 0 0) 0x7f6aa400cbb0 con 0x7f6aa8111d00 2023-10-12T02:13:32.411 INFO:teuthology.orchestra.run.smithi158.stdout:1 2023-10-12T02:13:32.412 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.410+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa8111d00 msgr2=0x7f6aa81ac150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.412 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.410+0000 7f6ab046d640 1 --2- 172.21.15.158:0/3919453285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa8111d00 0x7f6aa81ac150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.412 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.410+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 shutdown_connections 2023-10-12T02:13:32.413 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.410+0000 7f6ab046d640 1 --2- 172.21.15.158:0/3919453285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa8111d00 0x7f6aa81ac150 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.413 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.410+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 >> 172.21.15.158:0/3919453285 conn(0x7f6aa806d820 msgr2=0x7f6aa811bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.413 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.410+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 shutdown_connections 2023-10-12T02:13:32.413 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.410+0000 7f6ab046d640 1 -- 172.21.15.158:0/3919453285 wait complete. 2023-10-12T02:13:32.422 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new 8d72f668-655d-49a5-a5d0-90f0393c7a48 2 2023-10-12T02:13:32.567 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.561+0000 7f4660e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.569 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.566+0000 7f4660e2d640 1 Processor -- start 2023-10-12T02:13:32.569 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.566+0000 7f4660e2d640 1 -- start start 2023-10-12T02:13:32.570 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f4660e2d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f465c111d80 0x7f465c112190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.570 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f4660e2d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465c112950 con 0x7f465c116750 2023-10-12T02:13:32.570 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f4660e2d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c117290 con 0x7f465c111d80 2023-10-12T02:13:32.570 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f465a59c640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f465c111d80 0x7f465c112190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.570 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f4659d9b640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f465c116750 0x7f465c116b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.158:33024/0 (socket says 172.21.15.158:33024) 2023-10-12T02:13:32.571 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f4659d9b640 1 -- 172.21.15.158:0/1964078996 learned_addr learned my addr 172.21.15.158:0/1964078996 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-10-12T02:13:32.571 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f465a59c640 1 -- 172.21.15.158:0/1964078996 >> v1:172.21.15.125:6789/0 conn(0x7f465c116750 legacy=0x7f465c116b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.571 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.567+0000 7f465a59c640 1 -- 172.21.15.158:0/1964078996 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c117400 con 0x7f465c111d80 2023-10-12T02:13:32.571 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f465a59c640 1 --2- 172.21.15.158:0/1964078996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c112190 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e8fd0330d40e1a server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.571 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f465959a640 1 -- 172.21.15.158:0/1964078996 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4644009ce0 con 0x7f465c111d80 2023-10-12T02:13:32.572 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f465959a640 1 -- 172.21.15.158:0/1964078996 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4644009e40 con 0x7f465c111d80 2023-10-12T02:13:32.572 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f465959a640 1 -- 172.21.15.158:0/1964078996 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f464400e360 con 0x7f465c111d80 2023-10-12T02:13:32.574 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f4660e2d640 1 -- 172.21.15.158:0/1964078996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 msgr2=0x7f465c112190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.574 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f4660e2d640 1 --2- 172.21.15.158:0/1964078996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c112190 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.575 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f4660e2d640 1 -- 172.21.15.158:0/1964078996 shutdown_connections 2023-10-12T02:13:32.575 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f4660e2d640 1 --2- 172.21.15.158:0/1964078996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c112190 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.575 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.568+0000 7f4660e2d640 1 -- 172.21.15.158:0/1964078996 >> 172.21.15.158:0/1964078996 conn(0x7f465c06d7b0 msgr2=0x7f465c10fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.575 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.569+0000 7f4660e2d640 1 -- 172.21.15.158:0/1964078996 shutdown_connections 2023-10-12T02:13:32.576 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.569+0000 7f4660e2d640 1 -- 172.21.15.158:0/1964078996 wait complete. 2023-10-12T02:13:32.576 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.569+0000 7f4660e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.576 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.569+0000 7f4660e2d640 1 Processor -- start 2023-10-12T02:13:32.576 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.569+0000 7f4660e2d640 1 -- start start 2023-10-12T02:13:32.576 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.570+0000 7f4660e2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c1ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.576 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.570+0000 7f4660e2d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1ac9e0 con 0x7f465c111d80 2023-10-12T02:13:32.577 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.570+0000 7f465a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c1ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.577 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.570+0000 7f465a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c1ac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.158:54188/0 (socket says 172.21.15.158:54188) 2023-10-12T02:13:32.577 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.570+0000 7f465a59c640 1 -- 172.21.15.158:0/643901759 learned_addr learned my addr 172.21.15.158:0/643901759 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-10-12T02:13:32.577 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.570+0000 7f465a59c640 1 -- 172.21.15.158:0/643901759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c1acb20 con 0x7f465c111d80 2023-10-12T02:13:32.577 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.570+0000 7f465a59c640 1 --2- 172.21.15.158:0/643901759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c1ac410 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.578 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.571+0000 7f4652ffd640 1 -- 172.21.15.158:0/643901759 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4644014540 con 0x7f465c111d80 2023-10-12T02:13:32.578 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.571+0000 7f4652ffd640 1 -- 172.21.15.158:0/643901759 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f46440146a0 con 0x7f465c111d80 2023-10-12T02:13:32.578 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.571+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c1acd10 con 0x7f465c111d80 2023-10-12T02:13:32.578 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.571+0000 7f4652ffd640 1 -- 172.21.15.158:0/643901759 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4644014940 con 0x7f465c111d80 2023-10-12T02:13:32.578 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.571+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c1a6700 con 0x7f465c111d80 2023-10-12T02:13:32.579 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.571+0000 7f4652ffd640 1 -- 172.21.15.158:0/643901759 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f4644013400 con 0x7f465c111d80 2023-10-12T02:13:32.579 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.571+0000 7f4652ffd640 1 -- 172.21.15.158:0/643901759 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 977+0+0 (crc 0 0 0) 0x7f46440127d0 con 0x7f465c111d80 2023-10-12T02:13:32.579 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.572+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c1acf00 con 0x7f465c111d80 2023-10-12T02:13:32.580 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.574+0000 7f4652ffd640 1 -- 172.21.15.158:0/643901759 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7f4644012440 con 0x7f465c111d80 2023-10-12T02:13:32.647 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.646+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8d72f668-655d-49a5-a5d0-90f0393c7a48", "id": 2} v 0) v1 -- 0x7f465c116750 con 0x7f465c111d80 2023-10-12T02:13:32.648 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.647+0000 7f4652ffd640 1 -- 172.21.15.158:0/643901759 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d72f668-655d-49a5-a5d0-90f0393c7a48", "id": 2}]=0 v6) v1 ==== 112+0+2 (crc 0 0 0) 0x7f4644012a30 con 0x7f465c111d80 2023-10-12T02:13:32.649 INFO:teuthology.orchestra.run.smithi158.stdout:2 2023-10-12T02:13:32.650 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.648+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 msgr2=0x7f465c1ac410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.650 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.648+0000 7f4660e2d640 1 --2- 172.21.15.158:0/643901759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c1ac410 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.650 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.648+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 shutdown_connections 2023-10-12T02:13:32.650 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.648+0000 7f4660e2d640 1 --2- 172.21.15.158:0/643901759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f465c111d80 0x7f465c1ac410 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.651 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.648+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 >> 172.21.15.158:0/643901759 conn(0x7f465c06d7b0 msgr2=0x7f465c110350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.651 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.648+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 shutdown_connections 2023-10-12T02:13:32.651 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.648+0000 7f4660e2d640 1 -- 172.21.15.158:0/643901759 wait complete. 2023-10-12T02:13:32.660 DEBUG:teuthology.orchestra.run.smithi158:> sudo ceph --cluster ceph osd new fe84a5b4-f8b8-4604-adcb-34673b33cd52 3 2023-10-12T02:13:32.766 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.759+0000 7ff96bf9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.768 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.764+0000 7ff96bf9e640 1 Processor -- start 2023-10-12T02:13:32.768 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff96bf9e640 1 -- start start 2023-10-12T02:13:32.768 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff96bf9e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7ff96411a990 0x7ff96411cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.768 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff96bf9e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96411d380 con 0x7ff964111d80 2023-10-12T02:13:32.769 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff96bf9e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96411e520 con 0x7ff96411a990 2023-10-12T02:13:32.769 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff969539640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7ff96411a990 0x7ff96411cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.769 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff969d3a640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7ff964111d80 0x7ff964112190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.158:33040/0 (socket says 172.21.15.158:33040) 2023-10-12T02:13:32.769 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff969539640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7ff96411a990 0x7ff96411cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.158:54192/0 (socket says 172.21.15.158:54192) 2023-10-12T02:13:32.769 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.765+0000 7ff969d3a640 1 -- 172.21.15.158:0/231942001 learned_addr learned my addr 172.21.15.158:0/231942001 (peer_addr_for_me v1:172.21.15.158:0/0) 2023-10-12T02:13:32.770 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.766+0000 7ff969539640 1 -- 172.21.15.158:0/231942001 >> v1:172.21.15.125:6789/0 conn(0x7ff964111d80 legacy=0x7ff964112190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.770 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.766+0000 7ff969539640 1 -- 172.21.15.158:0/231942001 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96411e660 con 0x7ff96411a990 2023-10-12T02:13:32.770 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.766+0000 7ff969539640 1 --2- 172.21.15.158:0/231942001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96411a990 0x7ff96411cdb0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e7467c83ecbea4 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.771 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.766+0000 7f774c228f00 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-12T02:13:32.774 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.766+0000 7ff968d38640 1 -- 172.21.15.158:0/231942001 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff954009ce0 con 0x7ff96411a990 2023-10-12T02:13:32.774 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.766+0000 7ff968d38640 1 -- 172.21.15.158:0/231942001 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff954009e40 con 0x7ff96411a990 2023-10-12T02:13:32.774 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.766+0000 7ff968d38640 1 -- 172.21.15.158:0/231942001 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff95400e360 con 0x7ff96411a990 2023-10-12T02:13:32.774 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 1 -- 172.21.15.158:0/231942001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96411a990 msgr2=0x7ff96411cdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.775 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 1 --2- 172.21.15.158:0/231942001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96411a990 0x7ff96411cdb0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.775 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 1 -- 172.21.15.158:0/231942001 shutdown_connections 2023-10-12T02:13:32.775 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 1 --2- 172.21.15.158:0/231942001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96411a990 0x7ff96411cdb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.775 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 1 -- 172.21.15.158:0/231942001 >> 172.21.15.158:0/231942001 conn(0x7ff96406d7b0 msgr2=0x7ff96410fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.775 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 1 -- 172.21.15.158:0/231942001 shutdown_connections 2023-10-12T02:13:32.776 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 1 -- 172.21.15.158:0/231942001 wait complete. 2023-10-12T02:13:32.776 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.767+0000 7ff96bf9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:32.776 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.768+0000 7ff96bf9e640 1 Processor -- start 2023-10-12T02:13:32.776 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.768+0000 7ff96bf9e640 1 -- start start 2023-10-12T02:13:32.777 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.768+0000 7ff96bf9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff964111d80 0x7ff9641ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:32.777 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.768+0000 7ff96bf9e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641ac8a0 con 0x7ff964111d80 2023-10-12T02:13:32.777 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.768+0000 7ff969d3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff964111d80 0x7ff9641ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:32.777 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.768+0000 7ff969d3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff964111d80 0x7ff9641ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.158:54194/0 (socket says 172.21.15.158:54194) 2023-10-12T02:13:32.777 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.768+0000 7ff969d3a640 1 -- 172.21.15.158:0/1804127859 learned_addr learned my addr 172.21.15.158:0/1804127859 (peer_addr_for_me v2:172.21.15.158:0/0) 2023-10-12T02:13:32.778 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff969d3a640 1 -- 172.21.15.158:0/1804127859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9641ac9e0 con 0x7ff964111d80 2023-10-12T02:13:32.778 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff969d3a640 1 --2- 172.21.15.158:0/1804127859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff964111d80 0x7ff9641ac2d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:32.778 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff95a7fc640 1 -- 172.21.15.158:0/1804127859 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff9600059d0 con 0x7ff964111d80 2023-10-12T02:13:32.778 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff95a7fc640 1 -- 172.21.15.158:0/1804127859 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff960005b30 con 0x7ff964111d80 2023-10-12T02:13:32.778 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff95a7fc640 1 -- 172.21.15.158:0/1804127859 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff960005dd0 con 0x7ff964111d80 2023-10-12T02:13:32.779 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9641acc30 con 0x7ff964111d80 2023-10-12T02:13:32.779 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9641a6640 con 0x7ff964111d80 2023-10-12T02:13:32.779 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff95a7fc640 1 -- 172.21.15.158:0/1804127859 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7ff9600073b0 con 0x7ff964111d80 2023-10-12T02:13:32.779 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.769+0000 7ff95a7fc640 1 -- 172.21.15.158:0/1804127859 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1092+0+0 (crc 0 0 0) 0x7ff96000cc20 con 0x7ff964111d80 2023-10-12T02:13:32.779 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.770+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff964061d90 con 0x7ff964111d80 2023-10-12T02:13:32.780 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.772+0000 7ff95a7fc640 1 -- 172.21.15.158:0/1804127859 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7ff960007660 con 0x7ff964111d80 2023-10-12T02:13:32.845 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.844+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fe84a5b4-f8b8-4604-adcb-34673b33cd52", "id": 3} v 0) v1 -- 0x7ff9641a7100 con 0x7ff964111d80 2023-10-12T02:13:32.849 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.848+0000 7ff95a7fc640 1 -- 172.21.15.158:0/1804127859 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe84a5b4-f8b8-4604-adcb-34673b33cd52", "id": 3}]=0 v7) v1 ==== 112+0+2 (crc 0 0 0) 0x7ff96000c840 con 0x7ff964111d80 2023-10-12T02:13:32.850 INFO:teuthology.orchestra.run.smithi158.stdout:3 2023-10-12T02:13:32.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.849+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff964111d80 msgr2=0x7ff9641ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:32.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.849+0000 7ff96bf9e640 1 --2- 172.21.15.158:0/1804127859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff964111d80 0x7ff9641ac2d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.849+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 shutdown_connections 2023-10-12T02:13:32.851 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.849+0000 7ff96bf9e640 1 --2- 172.21.15.158:0/1804127859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff964111d80 0x7ff9641ac2d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:32.852 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.849+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 >> 172.21.15.158:0/1804127859 conn(0x7ff96406d7b0 msgr2=0x7ff96411afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:32.852 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.849+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 shutdown_connections 2023-10-12T02:13:32.852 INFO:teuthology.orchestra.run.smithi158.stderr:2023-10-12T02:13:32.849+0000 7ff96bf9e640 1 -- 172.21.15.158:0/1804127859 wait complete. 2023-10-12T02:13:32.861 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2023-10-12T02:13:32.899 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-12T02:13:32.901 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2023-10-12T02:13:32.902 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:32.898+0000 7f774c228f00 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-12T02:13:32.941 INFO:tasks.ceph.osd.0:Started 2023-10-12T02:13:32.942 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2023-10-12T02:13:32.978 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-12T02:13:32.979 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2023-10-12T02:13:33.017 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:33.015+0000 7f774c228f00 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-12T02:13:33.021 INFO:tasks.ceph.osd.1:Started 2023-10-12T02:13:33.022 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2023-10-12T02:13:33.055 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-12T02:13:33.055 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2023-10-12T02:13:33.065 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:33.063+0000 7f774c228f00 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-12T02:13:33.067 INFO:tasks.ceph.osd.2:Started 2023-10-12T02:13:33.067 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2023-10-12T02:13:33.100 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-12T02:13:33.100 DEBUG:teuthology.orchestra.run.smithi158:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2023-10-12T02:13:33.142 INFO:tasks.ceph.osd.3:Started 2023-10-12T02:13:33.142 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-12T02:13:33.174 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:33.172+0000 7f774c228f00 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-12T02:13:33.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.260+0000 7f1df722d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:33.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.265+0000 7f1df722d640 1 Processor -- start 2023-10-12T02:13:33.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.265+0000 7f1df722d640 1 -- start start 2023-10-12T02:13:33.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1df722d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f1df006cb00 0x7f1df012fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:33.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1df722d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df006c280 con 0x7f1df0130320 2023-10-12T02:13:33.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1df722d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df006c6c0 con 0x7f1df006cb00 2023-10-12T02:13:33.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1df4fc9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f1df006cb00 0x7f1df012fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:33.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1deffff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f1df0130320 0x7f1df0132cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:44536/0 (socket says 172.21.15.125:44536) 2023-10-12T02:13:33.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1deffff640 1 -- 172.21.15.125:0/2434920591 learned_addr learned my addr 172.21.15.125:0/2434920591 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:33.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1df4fc9640 1 -- 172.21.15.125:0/2434920591 >> v1:172.21.15.125:6789/0 conn(0x7f1df0130320 legacy=0x7f1df0132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:33.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.266+0000 7f1df4fc9640 1 -- 172.21.15.125:0/2434920591 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df006d8b0 con 0x7f1df006cb00 2023-10-12T02:13:33.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.267+0000 7f1df4fc9640 1 --2- 172.21.15.125:0/2434920591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df012fde0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83add5e004e750c server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:33.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.267+0000 7f1def7fe640 1 -- 172.21.15.125:0/2434920591 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1de4009ce0 con 0x7f1df006cb00 2023-10-12T02:13:33.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.267+0000 7f1def7fe640 1 -- 172.21.15.125:0/2434920591 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1de4009e40 con 0x7f1df006cb00 2023-10-12T02:13:33.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.267+0000 7f1def7fe640 1 -- 172.21.15.125:0/2434920591 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1de400e360 con 0x7f1df006cb00 2023-10-12T02:13:33.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 1 -- 172.21.15.125:0/2434920591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 msgr2=0x7f1df012fde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:33.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 1 --2- 172.21.15.125:0/2434920591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df012fde0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 1 -- 172.21.15.125:0/2434920591 shutdown_connections 2023-10-12T02:13:33.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 1 --2- 172.21.15.125:0/2434920591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df012fde0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 1 -- 172.21.15.125:0/2434920591 >> 172.21.15.125:0/2434920591 conn(0x7f1df00a3060 msgr2=0x7f1df00a3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:33.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 1 -- 172.21.15.125:0/2434920591 shutdown_connections 2023-10-12T02:13:33.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 1 -- 172.21.15.125:0/2434920591 wait complete. 2023-10-12T02:13:33.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.268+0000 7f1df722d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:33.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.269+0000 7f1df722d640 1 Processor -- start 2023-10-12T02:13:33.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.269+0000 7f1df722d640 1 -- start start 2023-10-12T02:13:33.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.270+0000 7f1df722d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df011c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:33.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.270+0000 7f1df722d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df011f5d0 con 0x7f1df006cb00 2023-10-12T02:13:33.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.270+0000 7f1df4fc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df011c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:33.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.270+0000 7f1df4fc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df011c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54314/0 (socket says 172.21.15.125:54314) 2023-10-12T02:13:33.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.270+0000 7f1df4fc9640 1 -- 172.21.15.125:0/2513984122 learned_addr learned my addr 172.21.15.125:0/2513984122 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:33.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1df4fc9640 1 -- 172.21.15.125:0/2513984122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df011f850 con 0x7f1df006cb00 2023-10-12T02:13:33.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1df4fc9640 1 --2- 172.21.15.125:0/2513984122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df011c460 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:33.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1ded7fa640 1 -- 172.21.15.125:0/2513984122 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1de4014620 con 0x7f1df006cb00 2023-10-12T02:13:33.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1ded7fa640 1 -- 172.21.15.125:0/2513984122 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1de4014780 con 0x7f1df006cb00 2023-10-12T02:13:33.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1ded7fa640 1 -- 172.21.15.125:0/2513984122 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1de4014a20 con 0x7f1df006cb00 2023-10-12T02:13:33.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df0120420 con 0x7f1df006cb00 2023-10-12T02:13:33.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df011ee50 con 0x7f1df006cb00 2023-10-12T02:13:33.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1ded7fa640 1 -- 172.21.15.125:0/2513984122 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f1de4011380 con 0x7f1df006cb00 2023-10-12T02:13:33.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.271+0000 7f1ded7fa640 1 -- 172.21.15.125:0/2513984122 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f1de4019040 con 0x7f1df006cb00 2023-10-12T02:13:33.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.272+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df0061d90 con 0x7f1df006cb00 2023-10-12T02:13:33.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.275+0000 7f1ded7fa640 1 -- 172.21.15.125:0/2513984122 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7f1de4011810 con 0x7f1df006cb00 2023-10-12T02:13:33.287 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:33.285+0000 7f774c228f00 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-12T02:13:33.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.337+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1df006c8e0 con 0x7f1df006cb00 2023-10-12T02:13:33.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.337+0000 7f1ded7fa640 1 -- 172.21.15.125:0/2513984122 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4151 (crc 0 0 0) 0x7f1de4011a60 con 0x7f1df006cb00 2023-10-12T02:13:33.339 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:33.339 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":7,"fsid":"bd5090fa-2483-41ce-a0be-4321207eb2da","created":"2023-10-12T02:13:30.743217+0000","modified":"2023-10-12T02:13:32.846139+0000","last_up_change":"0.000000","last_in_change":"2023-10-12T02:13:32.846139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e052e92d-fdbb-4870-b947-c74914b8c0a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8d72f668-655d-49a5-a5d0-90f0393c7a48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fe84a5b4-f8b8-4604-adcb-34673b33cd52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-12T02:13:33.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.338+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 msgr2=0x7f1df011c460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:33.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.338+0000 7f1df722d640 1 --2- 172.21.15.125:0/2513984122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df011c460 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.338+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 shutdown_connections 2023-10-12T02:13:33.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.338+0000 7f1df722d640 1 --2- 172.21.15.125:0/2513984122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1df006cb00 0x7f1df011c460 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.338+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 >> 172.21.15.125:0/2513984122 conn(0x7f1df00a3060 msgr2=0x7f1df00a1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:33.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.338+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 shutdown_connections 2023-10-12T02:13:33.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:33.338+0000 7f1df722d640 1 -- 172.21.15.125:0/2513984122 wait complete. 2023-10-12T02:13:33.351 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-12T02:13:33.351 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-12T02:13:33.454 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:33.452+0000 7f774c228f00 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-12T02:13:33.503 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:33.502+0000 7f774c228f00 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-12T02:13:33.683 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:33.681+0000 7f774c228f00 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-12T02:13:33.755 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-12T02:13:33.854 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.850+0000 7f3b68e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:33.855 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.854+0000 7f3b68e6d640 1 Processor -- start 2023-10-12T02:13:33.855 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.854+0000 7f3b68e6d640 1 -- start start 2023-10-12T02:13:33.858 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b68e6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3b641360d0 0x7f3b641384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:33.858 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b68e6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b6406cef0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.858 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b68e6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6409e5b0 con 0x7f3b641360d0 2023-10-12T02:13:33.858 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b61d9b640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3b641360d0 0x7f3b641384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:33.859 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b6259c640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f3b6412fcf0 0x7f3b64130100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:44540/0 (socket says 172.21.15.125:44540) 2023-10-12T02:13:33.859 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b6259c640 1 -- 172.21.15.125:0/1367404785 learned_addr learned my addr 172.21.15.125:0/1367404785 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:33.859 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b61d9b640 1 -- 172.21.15.125:0/1367404785 >> v1:172.21.15.125:6789/0 conn(0x7f3b6412fcf0 legacy=0x7f3b64130100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:33.859 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b61d9b640 1 -- 172.21.15.125:0/1367404785 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6412f8e0 con 0x7f3b641360d0 2023-10-12T02:13:33.860 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b61d9b640 1 --2- 172.21.15.125:0/1367404785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b641360d0 0x7f3b641384f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38479f10eb44795 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:33.860 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b6159a640 1 -- 172.21.15.125:0/1367404785 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b58009ce0 con 0x7f3b641360d0 2023-10-12T02:13:33.860 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b6159a640 1 -- 172.21.15.125:0/1367404785 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b58009e40 con 0x7f3b641360d0 2023-10-12T02:13:33.860 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.855+0000 7f3b6159a640 1 -- 172.21.15.125:0/1367404785 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b5800e360 con 0x7f3b641360d0 2023-10-12T02:13:33.860 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.856+0000 7f3b68e6d640 1 -- 172.21.15.125:0/1367404785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b641360d0 msgr2=0x7f3b641384f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:33.861 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.856+0000 7f3b68e6d640 1 --2- 172.21.15.125:0/1367404785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b641360d0 0x7f3b641384f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.861 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.856+0000 7f3b68e6d640 1 -- 172.21.15.125:0/1367404785 shutdown_connections 2023-10-12T02:13:33.861 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.856+0000 7f3b68e6d640 1 --2- 172.21.15.125:0/1367404785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b641360d0 0x7f3b641384f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.861 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.856+0000 7f3b68e6d640 1 -- 172.21.15.125:0/1367404785 >> 172.21.15.125:0/1367404785 conn(0x7f3b640a2290 msgr2=0x7f3b6409f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:33.862 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.856+0000 7f3b68e6d640 1 -- 172.21.15.125:0/1367404785 shutdown_connections 2023-10-12T02:13:33.862 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.856+0000 7f3b68e6d640 1 -- 172.21.15.125:0/1367404785 wait complete. 2023-10-12T02:13:33.862 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.857+0000 7f3b68e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:33.864 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.858+0000 7f3b68e6d640 1 Processor -- start 2023-10-12T02:13:33.865 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.858+0000 7f3b68e6d640 1 -- start start 2023-10-12T02:13:33.865 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.858+0000 7f3b68e6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6412fcf0 0x7f3b641c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:33.865 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.858+0000 7f3b68e6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6411d1d0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.865 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b6259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6412fcf0 0x7f3b641c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:33.865 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b6259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6412fcf0 0x7f3b641c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54326/0 (socket says 172.21.15.125:54326) 2023-10-12T02:13:33.866 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b6259c640 1 -- 172.21.15.125:0/3403550954 learned_addr learned my addr 172.21.15.125:0/3403550954 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:33.866 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b6259c640 1 -- 172.21.15.125:0/3403550954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b641c7890 con 0x7f3b6412fcf0 2023-10-12T02:13:33.866 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b6259c640 1 --2- 172.21.15.125:0/3403550954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6412fcf0 0x7f3b641c7d50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:33.866 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b52ffd640 1 -- 172.21.15.125:0/3403550954 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b540059d0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.867 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b52ffd640 1 -- 172.21.15.125:0/3403550954 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b54005b30 con 0x7f3b6412fcf0 2023-10-12T02:13:33.867 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6411c720 con 0x7f3b6412fcf0 2023-10-12T02:13:33.867 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b52ffd640 1 -- 172.21.15.125:0/3403550954 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b54005dd0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.867 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b64071b30 con 0x7f3b6412fcf0 2023-10-12T02:13:33.867 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b52ffd640 1 -- 172.21.15.125:0/3403550954 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f3b540073b0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.868 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.859+0000 7f3b52ffd640 1 -- 172.21.15.125:0/3403550954 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f3b5400f750 con 0x7f3b6412fcf0 2023-10-12T02:13:33.868 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.860+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6406cce0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.868 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.862+0000 7f3b52ffd640 1 -- 172.21.15.125:0/3403550954 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (crc 0 0 0) 0x7f3b6406cce0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.923 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.923+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b64061d90 con 0x7f3b6412fcf0 2023-10-12T02:13:33.924 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.923+0000 7f3b52ffd640 1 -- 172.21.15.125:0/3403550954 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4151 (crc 0 0 0) 0x7f3b5400f2e0 con 0x7f3b6412fcf0 2023-10-12T02:13:33.925 INFO:teuthology.misc.health.smithi125.stdout: 2023-10-12T02:13:33.925 INFO:teuthology.misc.health.smithi125.stdout:{"epoch":7,"fsid":"bd5090fa-2483-41ce-a0be-4321207eb2da","created":"2023-10-12T02:13:30.743217+0000","modified":"2023-10-12T02:13:32.846139+0000","last_up_change":"0.000000","last_in_change":"2023-10-12T02:13:32.846139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e052e92d-fdbb-4870-b947-c74914b8c0a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8d72f668-655d-49a5-a5d0-90f0393c7a48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fe84a5b4-f8b8-4604-adcb-34673b33cd52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-12T02:13:33.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.924+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6412fcf0 msgr2=0x7f3b641c7d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:33.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.924+0000 7f3b68e6d640 1 --2- 172.21.15.125:0/3403550954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6412fcf0 0x7f3b641c7d50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.924+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 shutdown_connections 2023-10-12T02:13:33.927 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.924+0000 7f3b68e6d640 1 --2- 172.21.15.125:0/3403550954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6412fcf0 0x7f3b641c7d50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:33.927 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.924+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 >> 172.21.15.125:0/3403550954 conn(0x7f3b640a2290 msgr2=0x7f3b64132b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:33.927 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.924+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 shutdown_connections 2023-10-12T02:13:33.927 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:33.924+0000 7f3b68e6d640 1 -- 172.21.15.125:0/3403550954 wait complete. 2023-10-12T02:13:33.934 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-10-12T02:13:34.597 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:34.595+0000 7f774c228f00 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-12T02:13:34.890 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:34.888+0000 7f774c228f00 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-12T02:13:34.946 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:34.944+0000 7f774c228f00 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-12T02:13:35.058 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:35.057+0000 7f774c228f00 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-12T02:13:36.082 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:13:36.080+0000 7f76ec327640 -1 mgr.server handle_report got status from non-daemon mon.a 2023-10-12T02:13:40.338 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-12T02:13:40.437 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.432+0000 7f81e906d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:40.438 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.436+0000 7f81e906d640 1 Processor -- start 2023-10-12T02:13:40.438 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.436+0000 7f81e906d640 1 -- start start 2023-10-12T02:13:40.440 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e906d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f81e4133c30 0x7f81e4138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:40.441 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e906d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e41330f0 con 0x7f81e406a180 2023-10-12T02:13:40.441 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e906d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e409a320 con 0x7f81e4133c30 2023-10-12T02:13:40.441 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81dbfff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f81e4133c30 0x7f81e4138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:40.441 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e2d9d640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f81e406a180 0x7f81e4133520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32798/0 (socket says 172.21.15.125:32798) 2023-10-12T02:13:40.441 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e2d9d640 1 -- 172.21.15.125:0/3186637066 learned_addr learned my addr 172.21.15.125:0/3186637066 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:40.442 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81dbfff640 1 -- 172.21.15.125:0/3186637066 >> v1:172.21.15.125:6789/0 conn(0x7f81e406a180 legacy=0x7f81e4133520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:40.442 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e259c640 1 -- 172.21.15.125:0/3186637066 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019149130 0 0) 0x7f81e41330f0 con 0x7f81e406a180 2023-10-12T02:13:40.442 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81dbfff640 1 -- 172.21.15.125:0/3186637066 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e409b460 con 0x7f81e4133c30 2023-10-12T02:13:40.442 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81dbfff640 1 --2- 172.21.15.125:0/3186637066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e4133c30 0x7f81e4138290 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18d0ea2e146155f server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:40.442 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e259c640 1 -- 172.21.15.125:0/3186637066 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81cc009ce0 con 0x7f81e4133c30 2023-10-12T02:13:40.443 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e259c640 1 -- 172.21.15.125:0/3186637066 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f81cc009e40 con 0x7f81e4133c30 2023-10-12T02:13:40.443 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.437+0000 7f81e259c640 1 -- 172.21.15.125:0/3186637066 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81cc00e360 con 0x7f81e4133c30 2023-10-12T02:13:40.443 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 1 -- 172.21.15.125:0/3186637066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e4133c30 msgr2=0x7f81e4138290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:40.443 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 1 --2- 172.21.15.125:0/3186637066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e4133c30 0x7f81e4138290 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.443 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 1 -- 172.21.15.125:0/3186637066 shutdown_connections 2023-10-12T02:13:40.444 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 1 --2- 172.21.15.125:0/3186637066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e4133c30 0x7f81e4138290 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.444 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 1 -- 172.21.15.125:0/3186637066 >> 172.21.15.125:0/3186637066 conn(0x7f81e40a1650 msgr2=0x7f81e40a1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:40.444 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 1 -- 172.21.15.125:0/3186637066 shutdown_connections 2023-10-12T02:13:40.444 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 1 -- 172.21.15.125:0/3186637066 wait complete. 2023-10-12T02:13:40.444 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.438+0000 7f81e906d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:40.446 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.439+0000 7f81e906d640 1 Processor -- start 2023-10-12T02:13:40.447 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.439+0000 7f81e906d640 1 -- start start 2023-10-12T02:13:40.447 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.440+0000 7f81e906d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e406a180 0x7f81e4097780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:40.447 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.440+0000 7f81e906d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40712a0 con 0x7f81e406a180 2023-10-12T02:13:40.447 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.440+0000 7f81e2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e406a180 0x7f81e4097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:40.447 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.440+0000 7f81e2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e406a180 0x7f81e4097780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49792/0 (socket says 172.21.15.125:49792) 2023-10-12T02:13:40.448 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.440+0000 7f81e2d9d640 1 -- 172.21.15.125:0/1517732777 learned_addr learned my addr 172.21.15.125:0/1517732777 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:40.448 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.440+0000 7f81e2d9d640 1 -- 172.21.15.125:0/1517732777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4071520 con 0x7f81e406a180 2023-10-12T02:13:40.448 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.441+0000 7f81e2d9d640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e406a180 0x7f81e4097780 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:40.448 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.441+0000 7f81db7fe640 1 -- 172.21.15.125:0/1517732777 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81d4004af0 con 0x7f81e406a180 2023-10-12T02:13:40.449 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.441+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e40724e0 con 0x7f81e406a180 2023-10-12T02:13:40.449 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.441+0000 7f81db7fe640 1 -- 172.21.15.125:0/1517732777 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f81d4004c50 con 0x7f81e406a180 2023-10-12T02:13:40.449 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.441+0000 7f81db7fe640 1 -- 172.21.15.125:0/1517732777 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81d4007120 con 0x7f81e406a180 2023-10-12T02:13:40.449 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.441+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e4072160 con 0x7f81e406a180 2023-10-12T02:13:40.449 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.442+0000 7f81db7fe640 1 -- 172.21.15.125:0/1517732777 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 4) v1 ==== 44799+0+0 (crc 0 0 0) 0x7f81d4007320 con 0x7f81e406a180 2023-10-12T02:13:40.450 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.442+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e4061d90 con 0x7f81e406a180 2023-10-12T02:13:40.450 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.442+0000 7f81db7fe640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f81c4037760 0x7f81c4039c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:40.450 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.442+0000 7f81db7fe640 1 -- 172.21.15.125:0/1517732777 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7f81d4046c50 con 0x7f81e406a180 2023-10-12T02:13:40.450 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.442+0000 7f81dbfff640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f81c4037760 0x7f81c4039c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:40.450 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.442+0000 7f81dbfff640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f81c4037760 0x7f81c4039c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:40.451 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.447+0000 7f81db7fe640 1 -- 172.21.15.125:0/1517732777 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f81d400f230 con 0x7f81e406a180 2023-10-12T02:13:40.626 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.624+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81e4067d20 con 0x7f81e406a180 2023-10-12T02:13:40.703 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.701+0000 7f81db7fe640 1 -- 172.21.15.125:0/1517732777 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6921 (crc 0 0 0) 0x7f81d40062e0 con 0x7f81e406a180 2023-10-12T02:13:40.704 INFO:teuthology.misc.health.smithi125.stdout: 2023-10-12T02:13:40.704 INFO:teuthology.misc.health.smithi125.stdout:{"epoch":11,"fsid":"bd5090fa-2483-41ce-a0be-4321207eb2da","created":"2023-10-12T02:13:30.743217+0000","modified":"2023-10-12T02:13:40.136163+0000","last_up_change":"2023-10-12T02:13:37.129684+0000","last_in_change":"2023-10-12T02:13:32.846139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-12T02:13:39.121616+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":121002782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6803","nonce":121002782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":121002782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6807","nonce":121002782}]},"public_addr":"172.21.15.125:6802/121002782","cluster_addr":"172.21.15.125:6803/121002782","heartbeat_back_addr":"172.21.15.125:6806/121002782","heartbeat_front_addr":"172.21.15.125:6807/121002782","state":["exists","up"]},{"osd":1,"uuid":"e052e92d-fdbb-4870-b947-c74914b8c0a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6801","nonce":2325473832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2325473832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2325473832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6805","nonce":2325473832}]},"public_addr":"172.21.15.125:6801/2325473832","cluster_addr":"172.21.15.125:6800/2325473832","heartbeat_back_addr":"172.21.15.125:6804/2325473832","heartbeat_front_addr":"172.21.15.125:6805/2325473832","state":["exists","up"]},{"osd":2,"uuid":"8d72f668-655d-49a5-a5d0-90f0393c7a48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6803","nonce":2109965471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":2109965471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6807","nonce":2109965471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":2109965471}]},"public_addr":"172.21.15.158:6803/2109965471","cluster_addr":"172.21.15.158:6802/2109965471","heartbeat_back_addr":"172.21.15.158:6807/2109965471","heartbeat_front_addr":"172.21.15.158:6806/2109965471","state":["exists","up"]},{"osd":3,"uuid":"fe84a5b4-f8b8-4604-adcb-34673b33cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":3388116905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6805","nonce":3388116905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6809","nonce":3388116905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3388116905}]},"public_addr":"172.21.15.158:6804/3388116905","cluster_addr":"172.21.15.158:6805/3388116905","heartbeat_back_addr":"172.21.15.158:6809/3388116905","heartbeat_front_addr":"172.21.15.158:6808/3388116905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-12T02:13:40.705 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f81c4037760 msgr2=0x7f81c4039c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:40.705 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f81c4037760 0x7f81c4039c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.706 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e406a180 msgr2=0x7f81e4097780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:40.706 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e406a180 0x7f81e4097780 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.706 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 shutdown_connections 2023-10-12T02:13:40.706 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f81c4037760 0x7f81c4039c70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.707 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 --2- 172.21.15.125:0/1517732777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81e406a180 0x7f81e4097780 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.707 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 >> 172.21.15.125:0/1517732777 conn(0x7f81e40a1650 msgr2=0x7f81e409dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:40.707 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 shutdown_connections 2023-10-12T02:13:40.707 INFO:teuthology.misc.health.smithi125.stderr:2023-10-12T02:13:40.703+0000 7f81e906d640 1 -- 172.21.15.125:0/1517732777 wait complete. 2023-10-12T02:13:40.715 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-10-12T02:13:40.715 INFO:tasks.ceph:Creating RBD pool 2023-10-12T02:13:40.716 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-12T02:13:40.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.816+0000 7effe699e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:40.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.824+0000 7effe699e640 1 Processor -- start 2023-10-12T02:13:40.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.824+0000 7effe699e640 1 -- start start 2023-10-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.824+0000 7effe699e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7effe0136100 0x7effe0138520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:40.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.824+0000 7effe699e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe006c210 con 0x7effe006d140 2023-10-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.824+0000 7effe699e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe006cdb0 con 0x7effe0136100 2023-10-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.824+0000 7effdf7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7effe0136100 0x7effe0138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdf7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7effe0136100 0x7effe0138520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49798/0 (socket says 172.21.15.125:49798) 2023-10-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdffff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7effe006d140 0x7effe0133440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32804/0 (socket says 172.21.15.125:32804) 2023-10-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdf7fe640 1 -- 172.21.15.125:0/2703801903 learned_addr learned my addr 172.21.15.125:0/2703801903 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:40.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdf7fe640 1 -- 172.21.15.125:0/2703801903 >> v1:172.21.15.125:6789/0 conn(0x7effe006d140 legacy=0x7effe0133440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdf7fe640 1 -- 172.21.15.125:0/2703801903 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe006d550 con 0x7effe0136100 2023-10-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdf7fe640 1 --2- 172.21.15.125:0/2703801903 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe0136100 0x7effe0138520 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e779fb7655068a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdeffd640 1 -- 172.21.15.125:0/2703801903 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effd4009dc0 con 0x7effe0136100 2023-10-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdeffd640 1 -- 172.21.15.125:0/2703801903 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7effd400e230 con 0x7effe0136100 2023-10-12T02:13:40.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.825+0000 7effdeffd640 1 -- 172.21.15.125:0/2703801903 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effd400e4d0 con 0x7effe0136100 2023-10-12T02:13:40.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 1 -- 172.21.15.125:0/2703801903 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe0136100 msgr2=0x7effe0138520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:40.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 1 --2- 172.21.15.125:0/2703801903 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe0136100 0x7effe0138520 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 1 -- 172.21.15.125:0/2703801903 shutdown_connections 2023-10-12T02:13:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 1 --2- 172.21.15.125:0/2703801903 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe0136100 0x7effe0138520 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 1 -- 172.21.15.125:0/2703801903 >> 172.21.15.125:0/2703801903 conn(0x7effe00a2d60 msgr2=0x7effe00a3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 1 -- 172.21.15.125:0/2703801903 shutdown_connections 2023-10-12T02:13:40.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 1 -- 172.21.15.125:0/2703801903 wait complete. 2023-10-12T02:13:40.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.826+0000 7effe699e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:40.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.827+0000 7effe699e640 1 Processor -- start 2023-10-12T02:13:40.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.827+0000 7effe699e640 1 -- start start 2023-10-12T02:13:40.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.828+0000 7effe699e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe006d140 0x7effe006fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:40.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.828+0000 7effe699e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0070eb0 con 0x7effe006d140 2023-10-12T02:13:40.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.828+0000 7effdffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe006d140 0x7effe006fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:40.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.828+0000 7effdffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe006d140 0x7effe006fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49812/0 (socket says 172.21.15.125:49812) 2023-10-12T02:13:40.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.828+0000 7effdffff640 1 -- 172.21.15.125:0/2522160522 learned_addr learned my addr 172.21.15.125:0/2522160522 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:40.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.828+0000 7effdffff640 1 -- 172.21.15.125:0/2522160522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe006f6f0 con 0x7effe006d140 2023-10-12T02:13:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.829+0000 7effdffff640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe006d140 0x7effe006fbb0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.829+0000 7effdcff9640 1 -- 172.21.15.125:0/2522160522 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effd00071e0 con 0x7effe006d140 2023-10-12T02:13:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.829+0000 7effdcff9640 1 -- 172.21.15.125:0/2522160522 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7effd0007340 con 0x7effe006d140 2023-10-12T02:13:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.829+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe011ca00 con 0x7effe006d140 2023-10-12T02:13:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.829+0000 7effdcff9640 1 -- 172.21.15.125:0/2522160522 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effd00075e0 con 0x7effe006d140 2023-10-12T02:13:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.829+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe011b310 con 0x7effe006d140 2023-10-12T02:13:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.830+0000 7effdcff9640 1 -- 172.21.15.125:0/2522160522 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 4) v1 ==== 44799+0+0 (crc 0 0 0) 0x7effd00057e0 con 0x7effe006d140 2023-10-12T02:13:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.830+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe0061d90 con 0x7effe006d140 2023-10-12T02:13:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.830+0000 7effdcff9640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7effcc037760 0x7effcc039c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.830+0000 7effdcff9640 1 -- 172.21.15.125:0/2522160522 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7effd00469b0 con 0x7effe006d140 2023-10-12T02:13:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.830+0000 7effdf7fe640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7effcc037760 0x7effcc039c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.831+0000 7effdf7fe640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7effcc037760 0x7effcc039c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:40.835+0000 7effdcff9640 1 -- 172.21.15.125:0/2522160522 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7effd000d330 con 0x7effe006d140 2023-10-12T02:13:41.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.002+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7effe004e1b0 con 0x7effe006d140 2023-10-12T02:13:41.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.139+0000 7effdcff9640 1 -- 172.21.15.125:0/2522160522 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v12) v1 ==== 109+0+0 (crc 0 0 0) 0x7effd000d150 con 0x7effe006d140 2023-10-12T02:13:41.142 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'rbd' created 2023-10-12T02:13:41.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7effcc037760 msgr2=0x7effcc039c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:41.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7effcc037760 0x7effcc039c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:41.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe006d140 msgr2=0x7effe006fbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:41.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe006d140 0x7effe006fbb0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 shutdown_connections 2023-10-12T02:13:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7effcc037760 0x7effcc039c70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 --2- 172.21.15.125:0/2522160522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe006d140 0x7effe006fbb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.141+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 >> 172.21.15.125:0/2522160522 conn(0x7effe00a2d60 msgr2=0x7effe00a3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.142+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 shutdown_connections 2023-10-12T02:13:41.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.142+0000 7effe699e640 1 -- 172.21.15.125:0/2522160522 wait complete. 2023-10-12T02:13:41.155 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-12T02:13:41.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.255+0000 7f6f2679e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:41.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.264+0000 7f6f2679e640 1 Processor -- start 2023-10-12T02:13:41.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f2679e640 1 -- start start 2023-10-12T02:13:41.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f2679e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f6f2006ca00 0x7f6f20134620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:41.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f2679e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f200a02b0 con 0x7f6f2006bee0 2023-10-12T02:13:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f2679e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200a0580 con 0x7f6f2006ca00 2023-10-12T02:13:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f1f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f6f2006ca00 0x7f6f20134620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f1f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f6f2006ca00 0x7f6f20134620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49818/0 (socket says 172.21.15.125:49818) 2023-10-12T02:13:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f1ffff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f6f2006bee0 0x7f6f2006c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32814/0 (socket says 172.21.15.125:32814) 2023-10-12T02:13:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.265+0000 7f6f1f7fe640 1 -- 172.21.15.125:0/265072693 learned_addr learned my addr 172.21.15.125:0/265072693 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.266+0000 7f6f1f7fe640 1 -- 172.21.15.125:0/265072693 >> v1:172.21.15.125:6789/0 conn(0x7f6f2006bee0 legacy=0x7f6f2006c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.266+0000 7f6f1f7fe640 1 -- 172.21.15.125:0/265072693 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2009a790 con 0x7f6f2006ca00 2023-10-12T02:13:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.266+0000 7f6f1f7fe640 1 --2- 172.21.15.125:0/265072693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006ca00 0x7f6f20134620 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375b2f05a96c65d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.266+0000 7f6f1effd640 1 -- 172.21.15.125:0/265072693 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6f10009dd0 con 0x7f6f2006ca00 2023-10-12T02:13:41.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.266+0000 7f6f1effd640 1 -- 172.21.15.125:0/265072693 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6f1000e230 con 0x7f6f2006ca00 2023-10-12T02:13:41.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.266+0000 7f6f1effd640 1 -- 172.21.15.125:0/265072693 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6f1000e4d0 con 0x7f6f2006ca00 2023-10-12T02:13:41.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 1 -- 172.21.15.125:0/265072693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006ca00 msgr2=0x7f6f20134620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:41.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 1 --2- 172.21.15.125:0/265072693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006ca00 0x7f6f20134620 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:41.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 1 -- 172.21.15.125:0/265072693 shutdown_connections 2023-10-12T02:13:41.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 1 --2- 172.21.15.125:0/265072693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006ca00 0x7f6f20134620 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:41.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 1 -- 172.21.15.125:0/265072693 >> 172.21.15.125:0/265072693 conn(0x7f6f2009f790 msgr2=0x7f6f2009fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:41.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 1 -- 172.21.15.125:0/265072693 shutdown_connections 2023-10-12T02:13:41.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 1 -- 172.21.15.125:0/265072693 wait complete. 2023-10-12T02:13:41.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.267+0000 7f6f2679e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:41.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.268+0000 7f6f2679e640 1 Processor -- start 2023-10-12T02:13:41.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.268+0000 7f6f2679e640 1 -- start start 2023-10-12T02:13:41.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f2679e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006bee0 0x7f6f2012ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:41.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f2679e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20117360 con 0x7f6f2006bee0 2023-10-12T02:13:41.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006bee0 0x7f6f2012ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:41.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006bee0 0x7f6f2012ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49824/0 (socket says 172.21.15.125:49824) 2023-10-12T02:13:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1ffff640 1 -- 172.21.15.125:0/3930425108 learned_addr learned my addr 172.21.15.125:0/3930425108 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1ffff640 1 -- 172.21.15.125:0/3930425108 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f20133b80 con 0x7f6f2006bee0 2023-10-12T02:13:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1ffff640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006bee0 0x7f6f2012ead0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6f080071e0 con 0x7f6f2006bee0 2023-10-12T02:13:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6f08007340 con 0x7f6f2006bee0 2023-10-12T02:13:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f20134290 con 0x7f6f2006bee0 2023-10-12T02:13:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6f080075e0 con 0x7f6f2006bee0 2023-10-12T02:13:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.269+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f20119000 con 0x7f6f2006bee0 2023-10-12T02:13:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.270+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 4) v1 ==== 44799+0+0 (crc 0 0 0) 0x7f6f080057e0 con 0x7f6f2006bee0 2023-10-12T02:13:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.270+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f201339e0 con 0x7f6f2006bee0 2023-10-12T02:13:41.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.270+0000 7f6f1cff9640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f6f000377b0 0x7f6f00039cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:41.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.270+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2661+0+0 (crc 0 0 0) 0x7f6f08046b40 con 0x7f6f2006bee0 2023-10-12T02:13:41.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.270+0000 7f6f1f7fe640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f6f000377b0 0x7f6f00039cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:41.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.271+0000 7f6f1f7fe640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f6f000377b0 0x7f6f00039cc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:41.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.275+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f6f201339e0 con 0x7f6f2006bee0 2023-10-12T02:13:41.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.440+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6f201339e0 con 0x7f6f2006bee0 2023-10-12T02:13:41.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:41.797+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f6f0800c6e0 con 0x7f6f2006bee0 2023-10-12T02:13:42.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.151+0000 7f6f1cff9640 1 -- 172.21.15.125:0/3930425108 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v13) v1 ==== 173+0+0 (crc 0 0 0) 0x7f6f0800e810 con 0x7f6f2006bee0 2023-10-12T02:13:42.153 INFO:teuthology.orchestra.run.smithi125.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-12T02:13:42.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f6f000377b0 msgr2=0x7f6f00039cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f6f000377b0 0x7f6f00039cc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006bee0 msgr2=0x7f6f2012ead0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006bee0 0x7f6f2012ead0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 shutdown_connections 2023-10-12T02:13:42.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f6f000377b0 0x7f6f00039cc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 --2- 172.21.15.125:0/3930425108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f2006bee0 0x7f6f2012ead0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 >> 172.21.15.125:0/3930425108 conn(0x7f6f2009f790 msgr2=0x7f6f2012c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:42.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 shutdown_connections 2023-10-12T02:13:42.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.153+0000 7f6f2679e640 1 -- 172.21.15.125:0/3930425108 wait complete. 2023-10-12T02:13:42.169 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-12T02:13:42.170 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-12T02:13:42.170 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-12T02:13:42.171 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-12T02:13:42.171 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-12T02:13:42.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.275+0000 7f5193e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:42.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.279+0000 7f5193e2d640 1 Processor -- start 2023-10-12T02:13:42.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.279+0000 7f5193e2d640 1 -- start start 2023-10-12T02:13:42.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f5193e2d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f518c06fdc0 0x7f518c1365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:42.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f5193e2d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c071490 con 0x7f518c070cb0 2023-10-12T02:13:42.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f5193e2d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c072040 con 0x7f518c06fdc0 2023-10-12T02:13:42.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f51913c8640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f518c06fdc0 0x7f518c1365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:42.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f5191bc9640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f518c070cb0 0x7f518c06f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32818/0 (socket says 172.21.15.125:32818) 2023-10-12T02:13:42.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f51913c8640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f518c06fdc0 0x7f518c1365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49838/0 (socket says 172.21.15.125:49838) 2023-10-12T02:13:42.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f5191bc9640 1 -- 172.21.15.125:0/3927341819 learned_addr learned my addr 172.21.15.125:0/3927341819 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:42.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f51913c8640 1 -- 172.21.15.125:0/3927341819 >> v1:172.21.15.125:6789/0 conn(0x7f518c070cb0 legacy=0x7f518c06f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f51913c8640 1 -- 172.21.15.125:0/3927341819 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c071180 con 0x7f518c06fdc0 2023-10-12T02:13:42.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.280+0000 7f51913c8640 1 --2- 172.21.15.125:0/3927341819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c1365b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f53a953fdd5a220 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:42.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5190bc7640 1 -- 172.21.15.125:0/3927341819 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f517c009ce0 con 0x7f518c06fdc0 2023-10-12T02:13:42.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5190bc7640 1 -- 172.21.15.125:0/3927341819 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f517c009e40 con 0x7f518c06fdc0 2023-10-12T02:13:42.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5190bc7640 1 -- 172.21.15.125:0/3927341819 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f517c00e360 con 0x7f518c06fdc0 2023-10-12T02:13:42.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5193e2d640 1 -- 172.21.15.125:0/3927341819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 msgr2=0x7f518c1365b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5193e2d640 1 --2- 172.21.15.125:0/3927341819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c1365b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5193e2d640 1 -- 172.21.15.125:0/3927341819 shutdown_connections 2023-10-12T02:13:42.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5193e2d640 1 --2- 172.21.15.125:0/3927341819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c1365b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5193e2d640 1 -- 172.21.15.125:0/3927341819 >> 172.21.15.125:0/3927341819 conn(0x7f518c0a14d0 msgr2=0x7f518c0a18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:42.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5193e2d640 1 -- 172.21.15.125:0/3927341819 shutdown_connections 2023-10-12T02:13:42.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.281+0000 7f5193e2d640 1 -- 172.21.15.125:0/3927341819 wait complete. 2023-10-12T02:13:42.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.282+0000 7f5193e2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:42.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.282+0000 7f5193e2d640 1 Processor -- start 2023-10-12T02:13:42.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.283+0000 7f5193e2d640 1 -- start start 2023-10-12T02:13:42.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.283+0000 7f5193e2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c095f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:42.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.283+0000 7f5193e2d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c097bf0 con 0x7f518c06fdc0 2023-10-12T02:13:42.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.283+0000 7f5191bc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c095f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:42.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.283+0000 7f5191bc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c095f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49850/0 (socket says 172.21.15.125:49850) 2023-10-12T02:13:42.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.283+0000 7f5191bc9640 1 -- 172.21.15.125:0/1010778234 learned_addr learned my addr 172.21.15.125:0/1010778234 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:42.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.283+0000 7f5191bc9640 1 -- 172.21.15.125:0/1010778234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0948b0 con 0x7f518c06fdc0 2023-10-12T02:13:42.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.284+0000 7f5191bc9640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c095f30 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:42.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.284+0000 7f51827fc640 1 -- 172.21.15.125:0/1010778234 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f51880071e0 con 0x7f518c06fdc0 2023-10-12T02:13:42.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.284+0000 7f51827fc640 1 -- 172.21.15.125:0/1010778234 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5188007340 con 0x7f518c06fdc0 2023-10-12T02:13:42.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.284+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c092fb0 con 0x7f518c06fdc0 2023-10-12T02:13:42.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.284+0000 7f51827fc640 1 -- 172.21.15.125:0/1010778234 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f51880075e0 con 0x7f518c06fdc0 2023-10-12T02:13:42.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.284+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c093730 con 0x7f518c06fdc0 2023-10-12T02:13:42.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.285+0000 7f51827fc640 1 -- 172.21.15.125:0/1010778234 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f5188005770 con 0x7f518c06fdc0 2023-10-12T02:13:42.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.285+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c061d90 con 0x7f518c06fdc0 2023-10-12T02:13:42.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.285+0000 7f51827fc640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5170037840 0x7f5170039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:42.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.285+0000 7f51827fc640 1 -- 172.21.15.125:0/1010778234 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5188046c10 con 0x7f518c06fdc0 2023-10-12T02:13:42.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.285+0000 7f51913c8640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5170037840 0x7f5170039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:42.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.286+0000 7f51913c8640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5170037840 0x7f5170039d50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:42.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.289+0000 7f51827fc640 1 -- 172.21.15.125:0/1010778234 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f518800d330 con 0x7f518c06fdc0 2023-10-12T02:13:42.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.503+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f518c06b820 con 0x7f518c06fdc0 2023-10-12T02:13:42.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.504+0000 7f51827fc640 1 -- 172.21.15.125:0/1010778234 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+86181 (crc 0 0 0) 0x7f518800d150 con 0x7f518c06fdc0 2023-10-12T02:13:42.508 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.506+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5170037840 msgr2=0x7f5170039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.506+0000 7f5193e2d640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5170037840 0x7f5170039d50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.506+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 msgr2=0x7f518c095f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.506+0000 7f5193e2d640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c095f30 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.507+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 shutdown_connections 2023-10-12T02:13:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.507+0000 7f5193e2d640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5170037840 0x7f5170039d50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.507+0000 7f5193e2d640 1 --2- 172.21.15.125:0/1010778234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f518c06fdc0 0x7f518c095f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.507+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 >> 172.21.15.125:0/1010778234 conn(0x7f518c0a14d0 msgr2=0x7f518c1b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.507+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 shutdown_connections 2023-10-12T02:13:42.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.507+0000 7f5193e2d640 1 -- 172.21.15.125:0/1010778234 wait complete. 2023-10-12T02:13:42.519 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":1559993426},{"type":"v1","addr":"172.21.15.158:6801","nonce":1559993426}]},"active_addr":"172.21.15.158:6801/1559993426","active_change":"2023-10-12T02:13:35.072434+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":1793721129}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":362283530}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":1421152509}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.158:0","nonce":2517483651}]}]} 2023-10-12T02:13:42.521 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-12T02:13:42.521 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-12T02:13:42.522 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-12T02:13:42.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.620+0000 7f7e4175e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:42.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.624+0000 7f7e4175e640 1 Processor -- start 2023-10-12T02:13:42.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.624+0000 7f7e4175e640 1 -- start start 2023-10-12T02:13:42.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e4175e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f7e3c12f5d0 0x7f7e3c12f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:42.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e4175e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c06e400 con 0x7f7e3c06bee0 2023-10-12T02:13:42.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e4175e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c09a630 con 0x7f7e3c12f5d0 2023-10-12T02:13:42.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e3affd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f7e3c12f5d0 0x7f7e3c12f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:42.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e3a7fc640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f7e3c06bee0 0x7f7e3c06c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32828/0 (socket says 172.21.15.125:32828) 2023-10-12T02:13:42.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e3affd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f7e3c12f5d0 0x7f7e3c12f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49854/0 (socket says 172.21.15.125:49854) 2023-10-12T02:13:42.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e3a7fc640 1 -- 172.21.15.125:0/2430946979 learned_addr learned my addr 172.21.15.125:0/2430946979 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:42.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e3affd640 1 -- 172.21.15.125:0/2430946979 >> v1:172.21.15.125:6789/0 conn(0x7f7e3c06bee0 legacy=0x7f7e3c06c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e3affd640 1 -- 172.21.15.125:0/2430946979 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c06dcb0 con 0x7f7e3c12f5d0 2023-10-12T02:13:42.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e39ffb640 1 -- 172.21.15.125:0/2430946979 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739284552 0 0) 0x7f7e3c06e400 con 0x7f7e3c06bee0 2023-10-12T02:13:42.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e3affd640 1 --2- 172.21.15.125:0/2430946979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c12f5d0 0x7f7e3c12f9e0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d718b4e9d1d1849 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:42.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e39ffb640 1 -- 172.21.15.125:0/2430946979 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e2c00ac70 con 0x7f7e3c12f5d0 2023-10-12T02:13:42.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e39ffb640 1 -- 172.21.15.125:0/2430946979 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7e2c00add0 con 0x7f7e3c12f5d0 2023-10-12T02:13:42.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.625+0000 7f7e39ffb640 1 -- 172.21.15.125:0/2430946979 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e2c00f470 con 0x7f7e3c12f5d0 2023-10-12T02:13:42.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 1 -- 172.21.15.125:0/2430946979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c12f5d0 msgr2=0x7f7e3c12f9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 1 --2- 172.21.15.125:0/2430946979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c12f5d0 0x7f7e3c12f9e0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 1 -- 172.21.15.125:0/2430946979 shutdown_connections 2023-10-12T02:13:42.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 1 --2- 172.21.15.125:0/2430946979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c12f5d0 0x7f7e3c12f9e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 1 -- 172.21.15.125:0/2430946979 >> 172.21.15.125:0/2430946979 conn(0x7f7e3c09f840 msgr2=0x7f7e3c09fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:42.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 1 -- 172.21.15.125:0/2430946979 shutdown_connections 2023-10-12T02:13:42.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 1 -- 172.21.15.125:0/2430946979 wait complete. 2023-10-12T02:13:42.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.626+0000 7f7e4175e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:42.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.627+0000 7f7e4175e640 1 Processor -- start 2023-10-12T02:13:42.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.627+0000 7f7e4175e640 1 -- start start 2023-10-12T02:13:42.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.628+0000 7f7e4175e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c06bee0 0x7f7e3c11df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:42.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.628+0000 7f7e4175e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c11db60 con 0x7f7e3c06bee0 2023-10-12T02:13:42.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.628+0000 7f7e3affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c06bee0 0x7f7e3c11df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:42.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.628+0000 7f7e3affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c06bee0 0x7f7e3c11df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49870/0 (socket says 172.21.15.125:49870) 2023-10-12T02:13:42.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.628+0000 7f7e3affd640 1 -- 172.21.15.125:0/1563074243 learned_addr learned my addr 172.21.15.125:0/1563074243 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:42.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.628+0000 7f7e3affd640 1 -- 172.21.15.125:0/1563074243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c1216e0 con 0x7f7e3c06bee0 2023-10-12T02:13:42.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.628+0000 7f7e3affd640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c06bee0 0x7f7e3c11df20 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:42.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.629+0000 7f7e237fe640 1 -- 172.21.15.125:0/1563074243 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e2c011550 con 0x7f7e3c06bee0 2023-10-12T02:13:42.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.629+0000 7f7e237fe640 1 -- 172.21.15.125:0/1563074243 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7e2c0116b0 con 0x7f7e3c06bee0 2023-10-12T02:13:42.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.629+0000 7f7e237fe640 1 -- 172.21.15.125:0/1563074243 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e2c011950 con 0x7f7e3c06bee0 2023-10-12T02:13:42.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.629+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c11d4b0 con 0x7f7e3c06bee0 2023-10-12T02:13:42.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.629+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c11f110 con 0x7f7e3c06bee0 2023-10-12T02:13:42.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.630+0000 7f7e237fe640 1 -- 172.21.15.125:0/1563074243 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f7e2c011b40 con 0x7f7e3c06bee0 2023-10-12T02:13:42.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.630+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c06e020 con 0x7f7e3c06bee0 2023-10-12T02:13:42.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.630+0000 7f7e237fe640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f7e28037840 0x7f7e28039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:42.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.630+0000 7f7e237fe640 1 -- 172.21.15.125:0/1563074243 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7e2c018040 con 0x7f7e3c06bee0 2023-10-12T02:13:42.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.630+0000 7f7e3a7fc640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f7e28037840 0x7f7e28039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:42.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.631+0000 7f7e3a7fc640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f7e28037840 0x7f7e28039d50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:42.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.634+0000 7f7e237fe640 1 -- 172.21.15.125:0/1563074243 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f7e2c00c6c0 con 0x7f7e3c06bee0 2023-10-12T02:13:42.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.800+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e3c11d960 con 0x7f7e3c06bee0 2023-10-12T02:13:42.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.801+0000 7f7e237fe640 1 -- 172.21.15.125:0/1563074243 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8680 (crc 0 0 0) 0x7f7e2c00c6c0 con 0x7f7e3c06bee0 2023-10-12T02:13:42.803 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:42.804 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":13,"fsid":"bd5090fa-2483-41ce-a0be-4321207eb2da","created":"2023-10-12T02:13:30.743217+0000","modified":"2023-10-12T02:13:42.145045+0000","last_up_change":"2023-10-12T02:13:37.129684+0000","last_in_change":"2023-10-12T02:13:32.846139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-12T02:13:39.121616+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-12T02:13:41.021139+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":121002782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6803","nonce":121002782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":121002782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6807","nonce":121002782}]},"public_addr":"172.21.15.125:6802/121002782","cluster_addr":"172.21.15.125:6803/121002782","heartbeat_back_addr":"172.21.15.125:6806/121002782","heartbeat_front_addr":"172.21.15.125:6807/121002782","state":["exists","up"]},{"osd":1,"uuid":"e052e92d-fdbb-4870-b947-c74914b8c0a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6801","nonce":2325473832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2325473832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2325473832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6805","nonce":2325473832}]},"public_addr":"172.21.15.125:6801/2325473832","cluster_addr":"172.21.15.125:6800/2325473832","heartbeat_back_addr":"172.21.15.125:6804/2325473832","heartbeat_front_addr":"172.21.15.125:6805/2325473832","state":["exists","up"]},{"osd":2,"uuid":"8d72f668-655d-49a5-a5d0-90f0393c7a48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6803","nonce":2109965471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":2109965471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6807","nonce":2109965471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":2109965471}]},"public_addr":"172.21.15.158:6803/2109965471","cluster_addr":"172.21.15.158:6802/2109965471","heartbeat_back_addr":"172.21.15.158:6807/2109965471","heartbeat_front_addr":"172.21.15.158:6806/2109965471","state":["exists","up"]},{"osd":3,"uuid":"fe84a5b4-f8b8-4604-adcb-34673b33cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":3388116905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6805","nonce":3388116905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6809","nonce":3388116905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3388116905}]},"public_addr":"172.21.15.158:6804/3388116905","cluster_addr":"172.21.15.158:6805/3388116905","heartbeat_back_addr":"172.21.15.158:6809/3388116905","heartbeat_front_addr":"172.21.15.158:6808/3388116905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-12T02:13:42.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f7e28037840 msgr2=0x7f7e28039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f7e28037840 0x7f7e28039d50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c06bee0 msgr2=0x7f7e3c11df20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:42.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c06bee0 0x7f7e3c11df20 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 shutdown_connections 2023-10-12T02:13:42.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f7e28037840 0x7f7e28039d50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 --2- 172.21.15.125:0/1563074243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c06bee0 0x7f7e3c11df20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:42.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 >> 172.21.15.125:0/1563074243 conn(0x7f7e3c09f840 msgr2=0x7f7e3c117000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:42.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 shutdown_connections 2023-10-12T02:13:42.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.803+0000 7f7e4175e640 1 -- 172.21.15.125:0/1563074243 wait complete. 2023-10-12T02:13:42.816 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-12T02:13:42.816 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-12T02:13:42.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.915+0000 7f29bcd9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.919+0000 7f29bcd9e640 1 Processor -- start 2023-10-12T02:13:43.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.919+0000 7f29bcd9e640 1 -- start start 2023-10-12T02:13:43.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29bcd9e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f29b806d800 0x7f29b812c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29bcd9e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b812c540 con 0x7f29b812ce40 2023-10-12T02:13:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29bcd9e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b806e990 con 0x7f29b806d800 2023-10-12T02:13:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b659c640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f29b806d800 0x7f29b812c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b659c640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f29b806d800 0x7f29b812c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49882/0 (socket says 172.21.15.125:49882) 2023-10-12T02:13:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29add9b640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f29b812ce40 0x7f29b812fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32834/0 (socket says 172.21.15.125:32834) 2023-10-12T02:13:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b659c640 1 -- 172.21.15.125:0/2199940657 learned_addr learned my addr 172.21.15.125:0/2199940657 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b659c640 1 -- 172.21.15.125:0/2199940657 >> v1:172.21.15.125:6789/0 conn(0x7f29b812ce40 legacy=0x7f29b812fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b659c640 1 -- 172.21.15.125:0/2199940657 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b806dd40 con 0x7f29b806d800 2023-10-12T02:13:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b659c640 1 --2- 172.21.15.125:0/2199940657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b812c900 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176b288e9ac1a94 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b5d9b640 1 -- 172.21.15.125:0/2199940657 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f29a8009dd0 con 0x7f29b806d800 2023-10-12T02:13:43.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b5d9b640 1 -- 172.21.15.125:0/2199940657 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f29a800e230 con 0x7f29b806d800 2023-10-12T02:13:43.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.920+0000 7f29b5d9b640 1 -- 172.21.15.125:0/2199940657 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f29a800e4d0 con 0x7f29b806d800 2023-10-12T02:13:43.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2199940657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 msgr2=0x7f29b812c900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 1 --2- 172.21.15.125:0/2199940657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b812c900 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2199940657 shutdown_connections 2023-10-12T02:13:43.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 1 --2- 172.21.15.125:0/2199940657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b812c900 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2199940657 >> 172.21.15.125:0/2199940657 conn(0x7f29b809f840 msgr2=0x7f29b809fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2199940657 shutdown_connections 2023-10-12T02:13:43.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2199940657 wait complete. 2023-10-12T02:13:43.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.921+0000 7f29bcd9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.922+0000 7f29bcd9e640 1 Processor -- start 2023-10-12T02:13:43.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.922+0000 7f29bcd9e640 1 -- start start 2023-10-12T02:13:43.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.923+0000 7f29bcd9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b8117000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.923+0000 7f29bcd9e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b8119390 con 0x7f29b806d800 2023-10-12T02:13:43.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.923+0000 7f29b659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b8117000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.923+0000 7f29b659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b8117000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49894/0 (socket says 172.21.15.125:49894) 2023-10-12T02:13:43.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.923+0000 7f29b659c640 1 -- 172.21.15.125:0/2977238823 learned_addr learned my addr 172.21.15.125:0/2977238823 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.923+0000 7f29b659c640 1 -- 172.21.15.125:0/2977238823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b811a050 con 0x7f29b806d800 2023-10-12T02:13:43.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.923+0000 7f29b659c640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b8117000 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.924+0000 7f29af7fe640 1 -- 172.21.15.125:0/2977238823 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f29a8014540 con 0x7f29b806d800 2023-10-12T02:13:43.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.924+0000 7f29af7fe640 1 -- 172.21.15.125:0/2977238823 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f29a80146a0 con 0x7f29b806d800 2023-10-12T02:13:43.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.924+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b8118920 con 0x7f29b806d800 2023-10-12T02:13:43.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.924+0000 7f29af7fe640 1 -- 172.21.15.125:0/2977238823 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f29a8014940 con 0x7f29b806d800 2023-10-12T02:13:43.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.924+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b8119620 con 0x7f29b806d800 2023-10-12T02:13:43.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.925+0000 7f29af7fe640 1 -- 172.21.15.125:0/2977238823 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f29a8014af0 con 0x7f29b806d800 2023-10-12T02:13:43.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.925+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b8061d90 con 0x7f29b806d800 2023-10-12T02:13:43.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.925+0000 7f29af7fe640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f2998037840 0x7f2998039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.925+0000 7f29af7fe640 1 -- 172.21.15.125:0/2977238823 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f29a8019040 con 0x7f29b806d800 2023-10-12T02:13:43.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.925+0000 7f29add9b640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f2998037840 0x7f2998039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.925+0000 7f29add9b640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f2998037840 0x7f2998039d50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:42.929+0000 7f29af7fe640 1 -- 172.21.15.125:0/2977238823 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f29a8018190 con 0x7f29b806d800 2023-10-12T02:13:43.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.095+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29b804e1b0 con 0x7f29b806d800 2023-10-12T02:13:43.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.096+0000 7f29af7fe640 1 -- 172.21.15.125:0/2977238823 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8680 (crc 0 0 0) 0x7f29a8010310 con 0x7f29b806d800 2023-10-12T02:13:43.098 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:43.098 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":13,"fsid":"bd5090fa-2483-41ce-a0be-4321207eb2da","created":"2023-10-12T02:13:30.743217+0000","modified":"2023-10-12T02:13:42.145045+0000","last_up_change":"2023-10-12T02:13:37.129684+0000","last_in_change":"2023-10-12T02:13:32.846139+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-12T02:13:39.121616+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-12T02:13:41.021139+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb1007ef-e5ea-4ae4-8b5c-e8e92bdac0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":121002782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6803","nonce":121002782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":121002782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6807","nonce":121002782}]},"public_addr":"172.21.15.125:6802/121002782","cluster_addr":"172.21.15.125:6803/121002782","heartbeat_back_addr":"172.21.15.125:6806/121002782","heartbeat_front_addr":"172.21.15.125:6807/121002782","state":["exists","up"]},{"osd":1,"uuid":"e052e92d-fdbb-4870-b947-c74914b8c0a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6801","nonce":2325473832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2325473832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2325473832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6805","nonce":2325473832}]},"public_addr":"172.21.15.125:6801/2325473832","cluster_addr":"172.21.15.125:6800/2325473832","heartbeat_back_addr":"172.21.15.125:6804/2325473832","heartbeat_front_addr":"172.21.15.125:6805/2325473832","state":["exists","up"]},{"osd":2,"uuid":"8d72f668-655d-49a5-a5d0-90f0393c7a48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6803","nonce":2109965471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":2109965471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6807","nonce":2109965471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":2109965471}]},"public_addr":"172.21.15.158:6803/2109965471","cluster_addr":"172.21.15.158:6802/2109965471","heartbeat_back_addr":"172.21.15.158:6807/2109965471","heartbeat_front_addr":"172.21.15.158:6806/2109965471","state":["exists","up"]},{"osd":3,"uuid":"fe84a5b4-f8b8-4604-adcb-34673b33cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":3388116905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6805","nonce":3388116905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6809","nonce":3388116905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":3388116905}]},"public_addr":"172.21.15.158:6804/3388116905","cluster_addr":"172.21.15.158:6805/3388116905","heartbeat_back_addr":"172.21.15.158:6809/3388116905","heartbeat_front_addr":"172.21.15.158:6808/3388116905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-12T02:13:43.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f2998037840 msgr2=0x7f2998039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f2998037840 0x7f2998039d50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 msgr2=0x7f29b8117000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b8117000 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 shutdown_connections 2023-10-12T02:13:43.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f2998037840 0x7f2998039d50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 --2- 172.21.15.125:0/2977238823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29b806d800 0x7f29b8117000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 >> 172.21.15.125:0/2977238823 conn(0x7f29b809f840 msgr2=0x7f29b8129760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 shutdown_connections 2023-10-12T02:13:43.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.098+0000 7f29bcd9e640 1 -- 172.21.15.125:0/2977238823 wait complete. 2023-10-12T02:13:43.111 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-12T02:13:43.112 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-12T02:13:43.112 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-12T02:13:43.113 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-12T02:13:43.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.221+0000 7f715ff5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.221+0000 7f5023d1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.221+0000 7f549fd5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.225+0000 7f5023d1e640 1 Processor -- start 2023-10-12T02:13:43.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.225+0000 7f715ff5e640 1 Processor -- start 2023-10-12T02:13:43.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.226+0000 7f5023d1e640 1 -- start start 2023-10-12T02:13:43.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.226+0000 7f715ff5e640 1 -- start start 2023-10-12T02:13:43.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.226+0000 7f549fd5e640 1 Processor -- start 2023-10-12T02:13:43.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.222+0000 7fbacbd9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.226+0000 7f549fd5e640 1 -- start start 2023-10-12T02:13:43.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.226+0000 7f5023d1e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f501c133d60 0x7f501c136180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.226+0000 7f715ff5e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f715806cb00 0x7f715812fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f5023d1e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f501c130a30 con 0x7f501c130df0 2023-10-12T02:13:43.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f5023d1e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f501c070840 con 0x7f501c133d60 2023-10-12T02:13:43.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715ff5e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715806c280 con 0x7f7158130320 2023-10-12T02:13:43.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715ff5e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715806c6c0 con 0x7f715806cb00 2023-10-12T02:13:43.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715dcfa640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f715806cb00 0x7f715812fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715dcfa640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f715806cb00 0x7f715812fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49914/0 (socket says 172.21.15.125:49914) 2023-10-12T02:13:43.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f50212b9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f501c133d60 0x7f501c136180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f50212b9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f501c133d60 0x7f501c136180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49902/0 (socket says 172.21.15.125:49902) 2023-10-12T02:13:43.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f50212b9640 1 -- 172.21.15.125:0/658614800 learned_addr learned my addr 172.21.15.125:0/658614800 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715dcfa640 1 -- 172.21.15.125:0/2870205684 learned_addr learned my addr 172.21.15.125:0/2870205684 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7fbacbd9e640 1 Processor -- start 2023-10-12T02:13:43.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f549fd5e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f549806c3c0 0x7f549812fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f549fd5e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549809ec10 con 0x7f549806cb40 2023-10-12T02:13:43.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f549fd5e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549812f7f0 con 0x7f549806c3c0 2023-10-12T02:13:43.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715dcfa640 1 -- 172.21.15.125:0/2870205684 >> v1:172.21.15.125:6789/0 conn(0x7f7158130320 legacy=0x7f7158132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f50212b9640 1 -- 172.21.15.125:0/658614800 >> v1:172.21.15.125:6789/0 conn(0x7f501c130df0 legacy=0x7f501c131200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715dcfa640 1 -- 172.21.15.125:0/2870205684 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715806d8b0 con 0x7f715806cb00 2023-10-12T02:13:43.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f50212b9640 1 -- 172.21.15.125:0/658614800 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c131910 con 0x7f501c133d60 2023-10-12T02:13:43.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7fbacbd9e640 1 -- start start 2023-10-12T02:13:43.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f715dcfa640 1 --2- 172.21.15.125:0/2870205684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715812fde0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b891a967e43ab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f549d2f9640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f549806cb40 0x7f5498130040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32860/0 (socket says 172.21.15.125:32860) 2023-10-12T02:13:43.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f50212b9640 1 --2- 172.21.15.125:0/658614800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c133d60 0x7f501c136180 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ebf414721e7a6c server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f549d2f9640 1 -- 172.21.15.125:0/1512952306 learned_addr learned my addr 172.21.15.125:0/1512952306 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:43.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.227+0000 7f549dafa640 1 --2- 172.21.15.125:0/1512952306 >> v2:172.21.15.125:3300/0 conn(0x7f549806c3c0 0x7f549812fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f715ccf8640 1 -- 172.21.15.125:0/2870205684 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7154009dd0 con 0x7f715806cb00 2023-10-12T02:13:43.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f5020ab8640 1 -- 172.21.15.125:0/658614800 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f500c009dd0 con 0x7f501c133d60 2023-10-12T02:13:43.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f5020ab8640 1 -- 172.21.15.125:0/658614800 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f500c00e230 con 0x7f501c133d60 2023-10-12T02:13:43.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f5020ab8640 1 -- 172.21.15.125:0/658614800 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f500c00e4d0 con 0x7f501c133d60 2023-10-12T02:13:43.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549caf8640 1 -- 172.21.15.125:0/1512952306 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007822138 0 0) 0x7f549809ec10 con 0x7f549806cb40 2023-10-12T02:13:43.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549caf8640 1 -- 172.21.15.125:0/1512952306 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54840034f0 con 0x7f549806cb40 2023-10-12T02:13:43.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549dafa640 1 -- 172.21.15.125:0/1512952306 >> v1:172.21.15.125:6789/0 conn(0x7f549806cb40 legacy=0x7f5498130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f715ccf8640 1 -- 172.21.15.125:0/2870205684 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f715400e230 con 0x7f715806cb00 2023-10-12T02:13:43.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f715ccf8640 1 -- 172.21.15.125:0/2870205684 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f715400e4d0 con 0x7f715806cb00 2023-10-12T02:13:43.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549dafa640 1 -- 172.21.15.125:0/1512952306 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498138ff0 con 0x7f549806c3c0 2023-10-12T02:13:43.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549dafa640 1 --2- 172.21.15.125:0/1512952306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549812fb00 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863701b271c72a50 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549caf8640 1 -- 172.21.15.125:0/1512952306 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5494009ce0 con 0x7f549806c3c0 2023-10-12T02:13:43.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549caf8640 1 -- 172.21.15.125:0/1512952306 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5494009e40 con 0x7f549806c3c0 2023-10-12T02:13:43.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7f549caf8640 1 -- 172.21.15.125:0/1512952306 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f549400e360 con 0x7f549806c3c0 2023-10-12T02:13:43.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7fbacbd9e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fbac406cd40 0x7fbac41332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7fbacbd9e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac406c480 con 0x7fbac4133810 2023-10-12T02:13:43.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7fbac9b3a640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fbac406cd40 0x7fbac41332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7fbac9339640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fbac4133810 0x7fbac412f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32874/0 (socket says 172.21.15.125:32874) 2023-10-12T02:13:43.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7fbac9339640 1 -- 172.21.15.125:0/1488420472 learned_addr learned my addr 172.21.15.125:0/1488420472 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:43.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.228+0000 7fbac9b3a640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fbac406cd40 0x7fbac41332d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49932/0 (socket says 172.21.15.125:49932) 2023-10-12T02:13:43.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7fbacbd9e640 1 -- 172.21.15.125:0/1488420472 --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac406d150 con 0x7fbac406cd40 2023-10-12T02:13:43.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7fbac9b3a640 1 -- 172.21.15.125:0/1488420472 >> v1:172.21.15.125:6789/0 conn(0x7fbac4133810 legacy=0x7fbac412f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 1 -- 172.21.15.125:0/658614800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c133d60 msgr2=0x7f501c136180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 1 --2- 172.21.15.125:0/658614800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c133d60 0x7f501c136180 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 1 -- 172.21.15.125:0/658614800 shutdown_connections 2023-10-12T02:13:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 1 --2- 172.21.15.125:0/658614800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c133d60 0x7f501c136180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 1 -- 172.21.15.125:0/658614800 >> 172.21.15.125:0/658614800 conn(0x7f501c09f950 msgr2=0x7f501c09fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7fbac9b3a640 1 -- 172.21.15.125:0/1488420472 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4132fc0 con 0x7fbac406cd40 2023-10-12T02:13:43.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 1 -- 172.21.15.125:0/2870205684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 msgr2=0x7f715812fde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2870205684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715812fde0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7fbac9b3a640 1 --2- 172.21.15.125:0/1488420472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac41332d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20556d568fa582f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 1 -- 172.21.15.125:0/2870205684 shutdown_connections 2023-10-12T02:13:43.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2870205684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715812fde0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 1 -- 172.21.15.125:0/2870205684 >> 172.21.15.125:0/2870205684 conn(0x7f71580a3060 msgr2=0x7f71580a3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 1 -- 172.21.15.125:0/658614800 shutdown_connections 2023-10-12T02:13:43.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 1 -- 172.21.15.125:0/658614800 wait complete. 2023-10-12T02:13:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7fbac8b38640 1 -- 172.21.15.125:0/1488420472 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbac000e040 con 0x7fbac406cd40 2023-10-12T02:13:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 1 -- 172.21.15.125:0/2870205684 shutdown_connections 2023-10-12T02:13:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 1 -- 172.21.15.125:0/2870205684 wait complete. 2023-10-12T02:13:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7fbac8b38640 1 -- 172.21.15.125:0/1488420472 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbac0007670 con 0x7fbac406cd40 2023-10-12T02:13:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7fbac8b38640 1 -- 172.21.15.125:0/1488420472 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbac0007910 con 0x7fbac406cd40 2023-10-12T02:13:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f549fd5e640 1 -- 172.21.15.125:0/1512952306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 msgr2=0x7f549812fb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f5023d1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f549fd5e640 1 --2- 172.21.15.125:0/1512952306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549812fb00 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.229+0000 7f715ff5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7f549fd5e640 1 -- 172.21.15.125:0/1512952306 shutdown_connections 2023-10-12T02:13:43.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7f549fd5e640 1 --2- 172.21.15.125:0/1512952306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549812fb00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7f549fd5e640 1 -- 172.21.15.125:0/1512952306 >> 172.21.15.125:0/1512952306 conn(0x7f549809fb60 msgr2=0x7f549809ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7f549fd5e640 1 -- 172.21.15.125:0/1512952306 shutdown_connections 2023-10-12T02:13:43.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7f549fd5e640 1 -- 172.21.15.125:0/1512952306 wait complete. 2023-10-12T02:13:43.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7f549fd5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7fbacbd9e640 1 -- 172.21.15.125:0/1488420472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 msgr2=0x7fbac41332d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/1488420472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac41332d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7fbacbd9e640 1 -- 172.21.15.125:0/1488420472 shutdown_connections 2023-10-12T02:13:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/1488420472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac41332d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7fbacbd9e640 1 -- 172.21.15.125:0/1488420472 >> 172.21.15.125:0/1488420472 conn(0x7fbac40a30d0 msgr2=0x7fbac40a34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7fbacbd9e640 1 -- 172.21.15.125:0/1488420472 shutdown_connections 2023-10-12T02:13:43.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.230+0000 7fbacbd9e640 1 -- 172.21.15.125:0/1488420472 wait complete. 2023-10-12T02:13:43.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f715ff5e640 1 Processor -- start 2023-10-12T02:13:43.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f5023d1e640 1 Processor -- start 2023-10-12T02:13:43.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f715ff5e640 1 -- start start 2023-10-12T02:13:43.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f5023d1e640 1 -- start start 2023-10-12T02:13:43.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7fbacbd9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f715ff5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715811c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f715ff5e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715811f5d0 con 0x7f715806cb00 2023-10-12T02:13:43.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f715dcfa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715811c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f715dcfa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715811c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49942/0 (socket says 172.21.15.125:49942) 2023-10-12T02:13:43.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f715dcfa640 1 -- 172.21.15.125:0/2314723209 learned_addr learned my addr 172.21.15.125:0/2314723209 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.231+0000 7f549fd5e640 1 Processor -- start 2023-10-12T02:13:43.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7fbacbd9e640 1 Processor -- start 2023-10-12T02:13:43.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f549fd5e640 1 -- start start 2023-10-12T02:13:43.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7fbacbd9e640 1 -- start start 2023-10-12T02:13:43.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f5023d1e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c130df0 0x7f501c1c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f715dcfa640 1 -- 172.21.15.125:0/2314723209 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715811f850 con 0x7f715806cb00 2023-10-12T02:13:43.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f5021aba640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c130df0 0x7f501c1c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f5021aba640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c130df0 0x7f501c1c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49950/0 (socket says 172.21.15.125:49950) 2023-10-12T02:13:43.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f715dcfa640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715811c460 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f5021aba640 1 -- 172.21.15.125:0/1862255117 learned_addr learned my addr 172.21.15.125:0/1862255117 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c1c54d0 con 0x7f501c130df0 2023-10-12T02:13:43.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7154014410 con 0x7f715806cb00 2023-10-12T02:13:43.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7154014570 con 0x7f715806cb00 2023-10-12T02:13:43.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7158120420 con 0x7f715806cb00 2023-10-12T02:13:43.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7154014810 con 0x7f715806cb00 2023-10-12T02:13:43.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715811ee50 con 0x7f715806cb00 2023-10-12T02:13:43.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7fbacbd9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac411c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.232+0000 7fbacbd9e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac411f090 con 0x7fbac406cd40 2023-10-12T02:13:43.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7fbac9b3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac411c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7fbac9b3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac411c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49966/0 (socket says 172.21.15.125:49966) 2023-10-12T02:13:43.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7fbac9b3a640 1 -- 172.21.15.125:0/38667971 learned_addr learned my addr 172.21.15.125:0/38667971 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f549fd5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f549fd5e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498070f30 con 0x7f549806c3c0 2023-10-12T02:13:43.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f549dafa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f549dafa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549806fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49968/0 (socket says 172.21.15.125:49968) 2023-10-12T02:13:43.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f549dafa640 1 -- 172.21.15.125:0/1448735921 learned_addr learned my addr 172.21.15.125:0/1448735921 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f5021aba640 1 -- 172.21.15.125:0/1862255117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c09c2b0 con 0x7f501c130df0 2023-10-12T02:13:43.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f5021aba640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c130df0 0x7f501c1c5a20 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f715812f750 con 0x7f715806cb00 2023-10-12T02:13:43.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f501800c8e0 con 0x7f501c130df0 2023-10-12T02:13:43.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7fbac9b3a640 1 -- 172.21.15.125:0/38667971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac411d400 con 0x7fbac406cd40 2023-10-12T02:13:43.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f501800ca40 con 0x7f501c130df0 2023-10-12T02:13:43.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f71540149c0 con 0x7f715806cb00 2023-10-12T02:13:43.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7fbac9b3a640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac411c1e0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f714e7fc640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f713c037840 0x7f713c039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.233+0000 7f549dafa640 1 -- 172.21.15.125:0/1448735921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549811d260 con 0x7f549806c3c0 2023-10-12T02:13:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbac000e040 con 0x7fbac406cd40 2023-10-12T02:13:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7154019040 con 0x7f715806cb00 2023-10-12T02:13:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac411edf0 con 0x7fbac406cd40 2023-10-12T02:13:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbac0012a90 con 0x7fbac406cd40 2023-10-12T02:13:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f501800cce0 con 0x7f501c130df0 2023-10-12T02:13:43.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f714e7fc640 1 --2- 172.21.15.125:0/2314723209 >> v2:172.21.15.125:6801/2325473832 conn(0x7f713c03c7e0 0x7f713c03ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 --> v2:172.21.15.125:6801/2325473832 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f713c03f300 con 0x7f713c03c7e0 2023-10-12T02:13:43.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f715404a860 con 0x7f715806cb00 2023-10-12T02:13:43.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f715d4f9640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f713c037840 0x7f713c039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f715d4f9640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f713c037840 0x7f713c039d50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbac0012d30 con 0x7fbac406cd40 2023-10-12T02:13:43.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac412d330 con 0x7fbac406cd40 2023-10-12T02:13:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f549dafa640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549806fa20 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5494014540 con 0x7f549806c3c0 2023-10-12T02:13:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.234+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f54940146a0 con 0x7f549806c3c0 2023-10-12T02:13:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5494014940 con 0x7f549806c3c0 2023-10-12T02:13:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7f549fd5e640 1 -- 172.21.15.125:0/1448735921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5498071680 con 0x7f549806c3c0 2023-10-12T02:13:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c11b490 con 0x7f501c130df0 2023-10-12T02:13:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c09c550 con 0x7f501c130df0 2023-10-12T02:13:43.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbac406bf50 con 0x7fbac406cd40 2023-10-12T02:13:43.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7f549fd5e640 1 -- 172.21.15.125:0/1448735921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549811bc40 con 0x7f549806c3c0 2023-10-12T02:13:43.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7f715e4fb640 1 --2- 172.21.15.125:0/2314723209 >> v2:172.21.15.125:6801/2325473832 conn(0x7f713c03c7e0 0x7f713c03ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-12T02:13:43.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7fbac00153f0 con 0x7fbac406cd40 2023-10-12T02:13:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.235+0000 7fbaba7fc640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fbaa80378e0 0x7fbaa8039df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fbac00485a0 con 0x7fbac406cd40 2023-10-12T02:13:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7fbaba7fc640 1 --2- 172.21.15.125:0/38667971 >> v2:172.21.15.158:6803/2109965471 conn(0x7fbaa803c880 0x7fbaa803ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 --> v2:172.21.15.158:6803/2109965471 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbaa803f3a0 con 0x7fbaa803c880 2023-10-12T02:13:43.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f501c09e830 con 0x7f501c130df0 2023-10-12T02:13:43.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7fbac9339640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fbaa80378e0 0x7fbaa8039df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7fbac00156f0 con 0x7fbac406cd40 2023-10-12T02:13:43.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7f549fd5e640 1 -- 172.21.15.125:0/1448735921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f549811c670 con 0x7f549806c3c0 2023-10-12T02:13:43.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7fbac9339640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fbaa80378e0 0x7fbaa8039df0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.236+0000 7fbaca33b640 1 --2- 172.21.15.125:0/38667971 >> v2:172.21.15.158:6803/2109965471 conn(0x7fbaa803c880 0x7fbaa803ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-12T02:13:43.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f5494014b40 con 0x7f549806c3c0 2023-10-12T02:13:43.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f5018012380 con 0x7f501c130df0 2023-10-12T02:13:43.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f548e7fc640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f547c037840 0x7f547c039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f50127fc640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5000037840 0x7f5000039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5494019040 con 0x7f549806c3c0 2023-10-12T02:13:43.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5018047660 con 0x7f501c130df0 2023-10-12T02:13:43.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f50127fc640 1 --2- 172.21.15.125:0/1862255117 >> v2:172.21.15.125:6802/121002782 conn(0x7f500003c7e0 0x7f500003ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f548e7fc640 1 --2- 172.21.15.125:0/1448735921 >> v2:172.21.15.158:6804/3388116905 conn(0x7f547c03c7e0 0x7f547c03ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 --> v2:172.21.15.158:6804/3388116905 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f547c03f300 con 0x7f547c03c7e0 2023-10-12T02:13:43.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f54940483d0 con 0x7f549806c3c0 2023-10-12T02:13:43.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 --> v2:172.21.15.125:6802/121002782 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f500003f300 con 0x7f500003c7e0 2023-10-12T02:13:43.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f5018047a40 con 0x7f501c130df0 2023-10-12T02:13:43.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f50212b9640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5000037840 0x7f5000039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f549d2f9640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f547c037840 0x7f547c039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f549d2f9640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f547c037840 0x7f547c039d50 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.237+0000 7f549e2fb640 1 --2- 172.21.15.125:0/1448735921 >> v2:172.21.15.158:6804/3388116905 conn(0x7f547c03c7e0 0x7f547c03ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-12T02:13:43.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.238+0000 7f50212b9640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5000037840 0x7f5000039d50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.238+0000 7f715e4fb640 1 --2- 172.21.15.125:0/2314723209 >> v2:172.21.15.125:6801/2325473832 conn(0x7f713c03c7e0 0x7f713c03ec70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.239+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== osd.1 v2:172.21.15.125:6801/2325473832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+4964 (crc 0 0 0) 0x7f713c03f300 con 0x7f713c03c7e0 2023-10-12T02:13:43.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.240+0000 7f50222bb640 1 --2- 172.21.15.125:0/1862255117 >> v2:172.21.15.125:6802/121002782 conn(0x7f500003c7e0 0x7f500003ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-12T02:13:43.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.240+0000 7f549e2fb640 1 --2- 172.21.15.125:0/1448735921 >> v2:172.21.15.158:6804/3388116905 conn(0x7f547c03c7e0 0x7f547c03ec70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.241+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== osd.3 v2:172.21.15.158:6804/3388116905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+4964 (crc 0 0 0) 0x7f547c03f300 con 0x7f547c03c7e0 2023-10-12T02:13:43.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.242+0000 7f50222bb640 1 --2- 172.21.15.125:0/1862255117 >> v2:172.21.15.125:6802/121002782 conn(0x7f500003c7e0 0x7f500003ec70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.243+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 --> v2:172.21.15.125:6801/2325473832 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f715812fad0 con 0x7f713c03c7e0 2023-10-12T02:13:43.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.243+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== osd.0 v2:172.21.15.125:6802/121002782 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+4964 (crc 0 0 0) 0x7f500003f300 con 0x7f500003c7e0 2023-10-12T02:13:43.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.243+0000 7fbaca33b640 1 --2- 172.21.15.125:0/38667971 >> v2:172.21.15.158:6803/2109965471 conn(0x7fbaa803c880 0x7fbaa803ed10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== osd.2 v2:172.21.15.158:6803/2109965471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+4964 (crc 0 0 0) 0x7fbaa803f3a0 con 0x7fbaa803c880 2023-10-12T02:13:43.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f714e7fc640 1 -- 172.21.15.125:0/2314723209 <== osd.1 v2:172.21.15.125:6801/2325473832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f715000a820 con 0x7f713c03c7e0 2023-10-12T02:13:43.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 >> v2:172.21.15.125:6801/2325473832 conn(0x7f713c03c7e0 msgr2=0x7f713c03ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2314723209 >> v2:172.21.15.125:6801/2325473832 conn(0x7f713c03c7e0 0x7f713c03ec70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f713c037840 msgr2=0x7f713c039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f713c037840 0x7f713c039d50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 msgr2=0x7f715811c460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715811c460 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.245+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 shutdown_connections 2023-10-12T02:13:43.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.246+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2314723209 >> v2:172.21.15.125:6801/2325473832 conn(0x7f713c03c7e0 0x7f713c03ec70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.246+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f713c037840 0x7f713c039d50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.246+0000 7f715ff5e640 1 --2- 172.21.15.125:0/2314723209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f715806cb00 0x7f715811c460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.246+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 >> 172.21.15.125:0/2314723209 conn(0x7f71580a3060 msgr2=0x7f71580a1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.246+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 shutdown_connections 2023-10-12T02:13:43.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.246+0000 7f715ff5e640 1 -- 172.21.15.125:0/2314723209 wait complete. 2023-10-12T02:13:43.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.247+0000 7f549fd5e640 1 -- 172.21.15.125:0/1448735921 --> v2:172.21.15.158:6804/3388116905 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f549806cd80 con 0x7f547c03c7e0 2023-10-12T02:13:43.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.247+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 --> v2:172.21.15.125:6802/121002782 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f501c0711f0 con 0x7f500003c7e0 2023-10-12T02:13:43.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f548e7fc640 1 -- 172.21.15.125:0/1448735921 <== osd.3 v2:172.21.15.158:6804/3388116905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f549806cd80 con 0x7f547c03c7e0 2023-10-12T02:13:43.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f50127fc640 1 -- 172.21.15.125:0/1862255117 <== osd.0 v2:172.21.15.125:6802/121002782 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f501400a820 con 0x7f500003c7e0 2023-10-12T02:13:43.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 >> v2:172.21.15.125:6802/121002782 conn(0x7f500003c7e0 msgr2=0x7f500003ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 --2- 172.21.15.125:0/1862255117 >> v2:172.21.15.125:6802/121002782 conn(0x7f500003c7e0 0x7f500003ec70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5000037840 msgr2=0x7f5000039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5000037840 0x7f5000039d50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 --> v2:172.21.15.158:6803/2109965471 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbac4061d90 con 0x7fbaa803c880 2023-10-12T02:13:43.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c130df0 msgr2=0x7f501c1c5a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c130df0 0x7f501c1c5a20 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 shutdown_connections 2023-10-12T02:13:43.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f5000037840 0x7f5000039d50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 -- 172.21.15.125:0/1448735921 >> v2:172.21.15.158:6804/3388116905 conn(0x7f547c03c7e0 msgr2=0x7f547c03ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.248+0000 7f5023d1e640 1 --2- 172.21.15.125:0/1862255117 >> v2:172.21.15.125:6802/121002782 conn(0x7f500003c7e0 0x7f500003ec70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f5023d1e640 1 --2- 172.21.15.125:0/1862255117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f501c130df0 0x7f501c1c5a20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 >> 172.21.15.125:0/1862255117 conn(0x7f501c09f950 msgr2=0x7f501c131d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 shutdown_connections 2023-10-12T02:13:43.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f5023d1e640 1 -- 172.21.15.125:0/1862255117 wait complete. 2023-10-12T02:13:43.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 --2- 172.21.15.125:0/1448735921 >> v2:172.21.15.158:6804/3388116905 conn(0x7f547c03c7e0 0x7f547c03ec70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 -- 172.21.15.125:0/1448735921 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f547c037840 msgr2=0x7f547c039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f547c037840 0x7f547c039d50 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 -- 172.21.15.125:0/1448735921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 msgr2=0x7f549806fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549806fa20 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 -- 172.21.15.125:0/1448735921 shutdown_connections 2023-10-12T02:13:43.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 --2- 172.21.15.125:0/1448735921 >> v2:172.21.15.158:6804/3388116905 conn(0x7f547c03c7e0 0x7f547c03ec70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f547c037840 0x7f547c039d50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 --2- 172.21.15.125:0/1448735921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f549806c3c0 0x7f549806fa20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7fbaba7fc640 1 -- 172.21.15.125:0/38667971 <== osd.2 v2:172.21.15.158:6803/2109965471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbac4061d90 con 0x7fbaa803c880 2023-10-12T02:13:43.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 -- 172.21.15.125:0/1448735921 >> 172.21.15.125:0/1448735921 conn(0x7f549809fb60 msgr2=0x7f54981374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 -- 172.21.15.125:0/1448735921 shutdown_connections 2023-10-12T02:13:43.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7f547bfff640 1 -- 172.21.15.125:0/1448735921 wait complete. 2023-10-12T02:13:43.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 >> v2:172.21.15.158:6803/2109965471 conn(0x7fbaa803c880 msgr2=0x7fbaa803ed10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.249+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/38667971 >> v2:172.21.15.158:6803/2109965471 conn(0x7fbaa803c880 0x7fbaa803ed10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fbaa80378e0 msgr2=0x7fbaa8039df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fbaa80378e0 0x7fbaa8039df0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 msgr2=0x7fbac411c1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac411c1e0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 shutdown_connections 2023-10-12T02:13:43.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/38667971 >> v2:172.21.15.158:6803/2109965471 conn(0x7fbaa803c880 0x7fbaa803ed10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fbaa80378e0 0x7fbaa8039df0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 --2- 172.21.15.125:0/38667971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbac406cd40 0x7fbac411c1e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 >> 172.21.15.125:0/38667971 conn(0x7fbac40a30d0 msgr2=0x7fbac412ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 shutdown_connections 2023-10-12T02:13:43.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.250+0000 7fbacbd9e640 1 -- 172.21.15.125:0/38667971 wait complete. 2023-10-12T02:13:43.324 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.324 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-12T02:13:43.325 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.326 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-12T02:13:43.327 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.327 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-12T02:13:43.328 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.328 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-12T02:13:43.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.473+0000 7f987a59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.473+0000 7f399faad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.473+0000 7f8f9ae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.473+0000 7f71f579e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.478+0000 7f399faad640 1 Processor -- start 2023-10-12T02:13:43.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.478+0000 7f8f9ae6d640 1 Processor -- start 2023-10-12T02:13:43.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.478+0000 7f987a59e640 1 Processor -- start 2023-10-12T02:13:43.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f399faad640 1 -- start start 2023-10-12T02:13:43.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f987a59e640 1 -- start start 2023-10-12T02:13:43.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f8f9ae6d640 1 -- start start 2023-10-12T02:13:43.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f71f579e640 1 Processor -- start 2023-10-12T02:13:43.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f71f579e640 1 -- start start 2023-10-12T02:13:43.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f987a59e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f9874133d60 0x7f9874136180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f987a59e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9874130a30 con 0x7f9874130df0 2023-10-12T02:13:43.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f987a59e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9874070840 con 0x7f9874133d60 2023-10-12T02:13:43.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f399faad640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f399812f5d0 0x7f399812f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f8f9ae6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f8f9412d010 0x7f8f94134d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f8f9ae6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9412c540 con 0x7f8f9406d800 2023-10-12T02:13:43.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f71f579e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f71f0133f00 0x7f71f0136320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f8f9ae6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9406e990 con 0x7f8f9412d010 2023-10-12T02:13:43.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f71f579e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f006de70 con 0x7f71f012d9f0 2023-10-12T02:13:43.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399d849640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f399812f5d0 0x7f399812f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399faad640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399806e400 con 0x7f399806bee0 2023-10-12T02:13:43.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399d048640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f399806bee0 0x7f399806c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32898/0 (socket says 172.21.15.125:32898) 2023-10-12T02:13:43.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f98737fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f9874133d60 0x7f9874136180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f98737fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f9874133d60 0x7f9874136180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49984/0 (socket says 172.21.15.125:49984) 2023-10-12T02:13:43.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f98737fe640 1 -- 172.21.15.125:0/1376065483 learned_addr learned my addr 172.21.15.125:0/1376065483 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399d048640 1 -- 172.21.15.125:0/655263191 learned_addr learned my addr 172.21.15.125:0/655263191 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:43.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f8f98c09640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f8f9406d800 0x7f8f9412c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32890/0 (socket says 172.21.15.125:32890) 2023-10-12T02:13:43.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f8f93fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f8f9412d010 0x7f8f94134d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399faad640 1 -- 172.21.15.125:0/655263191 --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399809a630 con 0x7f399812f5d0 2023-10-12T02:13:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.479+0000 7f71f579e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f012e510 con 0x7f71f0133f00 2023-10-12T02:13:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f71eeffd640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f71f012d9f0 0x7f71f012de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32894/0 (socket says 172.21.15.125:32894) 2023-10-12T02:13:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f71eeffd640 1 -- 172.21.15.125:0/2425849447 learned_addr learned my addr 172.21.15.125:0/2425849447 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f71e7fff640 1 --2- 172.21.15.125:0/2425849447 >> v2:172.21.15.125:3300/0 conn(0x7f71f0133f00 0x7f71f0136320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f8f98c09640 1 -- 172.21.15.125:0/2771688441 learned_addr learned my addr 172.21.15.125:0/2771688441 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:43.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f71e7fff640 1 -- 172.21.15.125:0/2425849447 >> v1:172.21.15.125:6789/0 conn(0x7f71f012d9f0 legacy=0x7f71f012de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f71e7fff640 1 -- 172.21.15.125:0/2425849447 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f006d660 con 0x7f71f0133f00 2023-10-12T02:13:43.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f8f93fff640 1 -- 172.21.15.125:0/2771688441 >> v1:172.21.15.125:6789/0 conn(0x7f8f9406d800 legacy=0x7f8f9412c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399c847640 1 -- 172.21.15.125:0/655263191 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171326208 0 0) 0x7f399806e400 con 0x7f399806bee0 2023-10-12T02:13:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399c847640 1 -- 172.21.15.125:0/655263191 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3984005700 con 0x7f399806bee0 2023-10-12T02:13:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f8f93fff640 1 -- 172.21.15.125:0/2771688441 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9412d420 con 0x7f8f9412d010 2023-10-12T02:13:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f9872ffd640 1 -- 172.21.15.125:0/1376065483 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617778186 0 0) 0x7f9874130a30 con 0x7f9874130df0 2023-10-12T02:13:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f9872ffd640 1 -- 172.21.15.125:0/1376065483 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f985c0034f0 con 0x7f9874130df0 2023-10-12T02:13:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399d849640 1 -- 172.21.15.125:0/655263191 >> v1:172.21.15.125:6789/0 conn(0x7f399806bee0 legacy=0x7f399806c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f98737fe640 1 -- 172.21.15.125:0/1376065483 >> v1:172.21.15.125:6789/0 conn(0x7f9874130df0 legacy=0x7f9874131200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399d849640 1 -- 172.21.15.125:0/655263191 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399806dcb0 con 0x7f399812f5d0 2023-10-12T02:13:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f98737fe640 1 -- 172.21.15.125:0/1376065483 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874131910 con 0x7f9874133d60 2023-10-12T02:13:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f8f93fff640 1 --2- 172.21.15.125:0/2771688441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9412d010 0x7f8f94134d80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46f434e979e6d0e server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f399d849640 1 --2- 172.21.15.125:0/655263191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399812f5d0 0x7f399812f9e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b5690df934f03b server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.480+0000 7f71e7fff640 1 --2- 172.21.15.125:0/2425849447 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f0133f00 0x7f71f0136320 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5725b5e1fcfa9951 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f98737fe640 1 --2- 172.21.15.125:0/1376065483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874133d60 0x7f9874136180 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48239b1492a7a205 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f71ee7fc640 1 -- 172.21.15.125:0/2425849447 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71d8009ce0 con 0x7f71f0133f00 2023-10-12T02:13:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f8f937fe640 1 -- 172.21.15.125:0/2771688441 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8f80010440 con 0x7f8f9412d010 2023-10-12T02:13:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f399c847640 1 -- 172.21.15.125:0/655263191 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3994009ce0 con 0x7f399812f5d0 2023-10-12T02:13:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f71ee7fc640 1 -- 172.21.15.125:0/2425849447 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f71d8009e40 con 0x7f71f0133f00 2023-10-12T02:13:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f71ee7fc640 1 -- 172.21.15.125:0/2425849447 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71d800e360 con 0x7f71f0133f00 2023-10-12T02:13:43.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f9872ffd640 1 -- 172.21.15.125:0/1376065483 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9868009dd0 con 0x7f9874133d60 2023-10-12T02:13:43.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f8f937fe640 1 -- 172.21.15.125:0/2771688441 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8f800105a0 con 0x7f8f9412d010 2023-10-12T02:13:43.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f8f937fe640 1 -- 172.21.15.125:0/2771688441 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8f80010840 con 0x7f8f9412d010 2023-10-12T02:13:43.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f399c847640 1 -- 172.21.15.125:0/655263191 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3994009e40 con 0x7f399812f5d0 2023-10-12T02:13:43.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f399c847640 1 -- 172.21.15.125:0/655263191 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3994009ce0 con 0x7f399812f5d0 2023-10-12T02:13:43.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f9872ffd640 1 -- 172.21.15.125:0/1376065483 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f986800e230 con 0x7f9874133d60 2023-10-12T02:13:43.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.481+0000 7f9872ffd640 1 -- 172.21.15.125:0/1376065483 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f986800e4d0 con 0x7f9874133d60 2023-10-12T02:13:43.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/2771688441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9412d010 msgr2=0x7f8f94134d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 1 --2- 172.21.15.125:0/2771688441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9412d010 0x7f8f94134d80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 1 -- 172.21.15.125:0/655263191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399812f5d0 msgr2=0x7f399812f9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 1 --2- 172.21.15.125:0/655263191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399812f5d0 0x7f399812f9e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/2771688441 shutdown_connections 2023-10-12T02:13:43.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 1 --2- 172.21.15.125:0/2771688441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9412d010 0x7f8f94134d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/2771688441 >> 172.21.15.125:0/2771688441 conn(0x7f8f9409f840 msgr2=0x7f8f9409fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 1 -- 172.21.15.125:0/655263191 shutdown_connections 2023-10-12T02:13:43.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 1 --2- 172.21.15.125:0/655263191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399812f5d0 0x7f399812f9e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 1 -- 172.21.15.125:0/655263191 >> 172.21.15.125:0/655263191 conn(0x7f399809f840 msgr2=0x7f399809fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 1 -- 172.21.15.125:0/2425849447 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f0133f00 msgr2=0x7f71f0136320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/2771688441 shutdown_connections 2023-10-12T02:13:43.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 1 --2- 172.21.15.125:0/2425849447 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f0133f00 0x7f71f0136320 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/2771688441 wait complete. 2023-10-12T02:13:43.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 1 -- 172.21.15.125:0/655263191 shutdown_connections 2023-10-12T02:13:43.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 1 -- 172.21.15.125:0/2425849447 shutdown_connections 2023-10-12T02:13:43.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 1 --2- 172.21.15.125:0/2425849447 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f0133f00 0x7f71f0136320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 1 -- 172.21.15.125:0/2425849447 >> 172.21.15.125:0/2425849447 conn(0x7f71f009ff80 msgr2=0x7f71f00a0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 1 -- 172.21.15.125:0/655263191 wait complete. 2023-10-12T02:13:43.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 1 -- 172.21.15.125:0/2425849447 shutdown_connections 2023-10-12T02:13:43.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 1 -- 172.21.15.125:0/2425849447 wait complete. 2023-10-12T02:13:43.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f987a59e640 1 -- 172.21.15.125:0/1376065483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874133d60 msgr2=0x7f9874136180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f987a59e640 1 --2- 172.21.15.125:0/1376065483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874133d60 0x7f9874136180 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f8f9ae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f987a59e640 1 -- 172.21.15.125:0/1376065483 shutdown_connections 2023-10-12T02:13:43.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f987a59e640 1 --2- 172.21.15.125:0/1376065483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874133d60 0x7f9874136180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f399faad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f987a59e640 1 -- 172.21.15.125:0/1376065483 >> 172.21.15.125:0/1376065483 conn(0x7f987409f950 msgr2=0x7f987409fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f71f579e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f987a59e640 1 -- 172.21.15.125:0/1376065483 shutdown_connections 2023-10-12T02:13:43.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.482+0000 7f987a59e640 1 -- 172.21.15.125:0/1376065483 wait complete. 2023-10-12T02:13:43.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.483+0000 7f987a59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.483+0000 7f71f579e640 1 Processor -- start 2023-10-12T02:13:43.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.483+0000 7f71f579e640 1 -- start start 2023-10-12T02:13:43.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f987a59e640 1 Processor -- start 2023-10-12T02:13:43.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f987a59e640 1 -- start start 2023-10-12T02:13:43.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f399faad640 1 Processor -- start 2023-10-12T02:13:43.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f8f9ae6d640 1 Processor -- start 2023-10-12T02:13:43.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f399faad640 1 -- start start 2023-10-12T02:13:43.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f8f9ae6d640 1 -- start start 2023-10-12T02:13:43.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f71f579e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f012d9f0 0x7f71f01c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f71f579e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0071e70 con 0x7f71f012d9f0 2023-10-12T02:13:43.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f71eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f012d9f0 0x7f71f01c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f71eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f012d9f0 0x7f71f01c3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50004/0 (socket says 172.21.15.125:50004) 2023-10-12T02:13:43.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f71eeffd640 1 -- 172.21.15.125:0/3571261763 learned_addr learned my addr 172.21.15.125:0/3571261763 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.484+0000 7f71eeffd640 1 -- 172.21.15.125:0/3571261763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f012d630 con 0x7f71f012d9f0 2023-10-12T02:13:43.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f987a59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874130df0 0x7f987411b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f71eeffd640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f012d9f0 0x7f71f01c3440 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f987a59e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987411ab20 con 0x7f9874130df0 2023-10-12T02:13:43.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f71e77fe640 1 -- 172.21.15.125:0/3571261763 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71e00059d0 con 0x7f71f012d9f0 2023-10-12T02:13:43.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f9873fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874130df0 0x7f987411b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f9873fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874130df0 0x7f987411b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50006/0 (socket says 172.21.15.125:50006) 2023-10-12T02:13:43.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f9873fff640 1 -- 172.21.15.125:0/1858974398 learned_addr learned my addr 172.21.15.125:0/1858974398 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f71e77fe640 1 -- 172.21.15.125:0/3571261763 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f71e0005b30 con 0x7f71f012d9f0 2023-10-12T02:13:43.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f006fe00 con 0x7f71f012d9f0 2023-10-12T02:13:43.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f71e77fe640 1 -- 172.21.15.125:0/3571261763 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71e0005dd0 con 0x7f71f012d9f0 2023-10-12T02:13:43.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f9873fff640 1 -- 172.21.15.125:0/1858974398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987411b0f0 con 0x7f9874130df0 2023-10-12T02:13:43.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f399faad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399806bee0 0x7f399811e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f399faad640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399811db40 con 0x7f399806bee0 2023-10-12T02:13:43.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f399d849640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399806bee0 0x7f399811e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f8f9ae6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9406d800 0x7f8f94117000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f399d849640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399806bee0 0x7f399811e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50022/0 (socket says 172.21.15.125:50022) 2023-10-12T02:13:43.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f399d849640 1 -- 172.21.15.125:0/52546056 learned_addr learned my addr 172.21.15.125:0/52546056 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f8f9ae6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f941180a0 con 0x7f8f9406d800 2023-10-12T02:13:43.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f0071560 con 0x7f71f012d9f0 2023-10-12T02:13:43.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f8f98c09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9406d800 0x7f8f94117000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f9873fff640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874130df0 0x7f987411b640 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.485+0000 7f8f98c09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9406d800 0x7f8f94117000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50036/0 (socket says 172.21.15.125:50036) 2023-10-12T02:13:43.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f8f98c09640 1 -- 172.21.15.125:0/470810834 learned_addr learned my addr 172.21.15.125:0/470810834 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f399d849640 1 -- 172.21.15.125:0/52546056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998121770 con 0x7f399806bee0 2023-10-12T02:13:43.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f9870ff9640 1 -- 172.21.15.125:0/1858974398 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f986400c940 con 0x7f9874130df0 2023-10-12T02:13:43.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f9870ff9640 1 -- 172.21.15.125:0/1858974398 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f986400caa0 con 0x7f9874130df0 2023-10-12T02:13:43.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f9870ff9640 1 -- 172.21.15.125:0/1858974398 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f986400cd40 con 0x7f9874130df0 2023-10-12T02:13:43.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987411d040 con 0x7f9874130df0 2023-10-12T02:13:43.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987411eb10 con 0x7f9874130df0 2023-10-12T02:13:43.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f8f98c09640 1 -- 172.21.15.125:0/470810834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94118920 con 0x7f8f9406d800 2023-10-12T02:13:43.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f399d849640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399806bee0 0x7f399811e000 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.486+0000 7f8f98c09640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9406d800 0x7f8f94117000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f398e7fc640 1 -- 172.21.15.125:0/52546056 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3994009ce0 con 0x7f399806bee0 2023-10-12T02:13:43.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f398e7fc640 1 -- 172.21.15.125:0/52546056 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3994014700 con 0x7f399806bee0 2023-10-12T02:13:43.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f398e7fc640 1 -- 172.21.15.125:0/52546056 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f39940149a0 con 0x7f399806bee0 2023-10-12T02:13:43.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f399faad640 1 -- 172.21.15.125:0/52546056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399811fbd0 con 0x7f399806bee0 2023-10-12T02:13:43.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f399faad640 1 -- 172.21.15.125:0/52546056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399811fef0 con 0x7f399806bee0 2023-10-12T02:13:43.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f71e77fe640 1 -- 172.21.15.125:0/3571261763 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f71e00070b0 con 0x7f71f012d9f0 2023-10-12T02:13:43.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f006fc00 con 0x7f71f012d9f0 2023-10-12T02:13:43.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f71e77fe640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f71d0037890 0x7f71d0039da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f71e77fe640 1 -- 172.21.15.125:0/3571261763 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f71e0046cd0 con 0x7f71f012d9f0 2023-10-12T02:13:43.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f9870ff9640 1 -- 172.21.15.125:0/1858974398 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f9864011310 con 0x7f9874130df0 2023-10-12T02:13:43.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987411aef0 con 0x7f9874130df0 2023-10-12T02:13:43.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f8f917fa640 1 -- 172.21.15.125:0/470810834 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8f880059d0 con 0x7f8f9406d800 2023-10-12T02:13:43.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f8f917fa640 1 -- 172.21.15.125:0/470810834 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8f88005b30 con 0x7f8f9406d800 2023-10-12T02:13:43.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f94119970 con 0x7f8f9406d800 2023-10-12T02:13:43.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f9870ff9640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f9860037840 0x7f9860039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f71e7fff640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f71d0037890 0x7f71d0039da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f9870ff9640 1 -- 172.21.15.125:0/1858974398 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f98640472e0 con 0x7f9874130df0 2023-10-12T02:13:43.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f8f917fa640 1 -- 172.21.15.125:0/470810834 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8f88005dd0 con 0x7f8f9406d800 2023-10-12T02:13:43.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.487+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9411a250 con 0x7f8f9406d800 2023-10-12T02:13:43.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.488+0000 7f98737fe640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f9860037840 0x7f9860039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.488+0000 7f71e7fff640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f71d0037890 0x7f71d0039da0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.488+0000 7f98737fe640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f9860037840 0x7f9860039d50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.489+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3974005010 con 0x7f399806bee0 2023-10-12T02:13:43.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.489+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94119570 con 0x7f8f9406d800 2023-10-12T02:13:43.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.489+0000 7f398e7fc640 1 -- 172.21.15.125:0/52546056 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f3994014b50 con 0x7f399806bee0 2023-10-12T02:13:43.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.489+0000 7f398e7fc640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f397c037840 0x7f397c039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.489+0000 7f399d048640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f397c037840 0x7f397c039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.489+0000 7f399d048640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f397c037840 0x7f397c039d50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.493+0000 7f71e77fe640 1 -- 172.21.15.125:0/3571261763 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f71e000d330 con 0x7f71f012d9f0 2023-10-12T02:13:43.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.497+0000 7f9870ff9640 1 -- 172.21.15.125:0/1858974398 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f986400db20 con 0x7f9874130df0 2023-10-12T02:13:43.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.497+0000 7f398e7fc640 1 -- 172.21.15.125:0/52546056 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3994019040 con 0x7f399806bee0 2023-10-12T02:13:43.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.498+0000 7f8f917fa640 1 -- 172.21.15.125:0/470810834 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f8f880070b0 con 0x7f8f9406d800 2023-10-12T02:13:43.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.498+0000 7f8f917fa640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f8f78037890 0x7f8f78039da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.499+0000 7f8f93fff640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f8f78037890 0x7f8f78039da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.499+0000 7f8f93fff640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f8f78037890 0x7f8f78039da0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.502+0000 7f8f917fa640 1 -- 172.21.15.125:0/470810834 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8f88010e30 con 0x7f8f9406d800 2023-10-12T02:13:43.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.502+0000 7f398e7fc640 1 -- 172.21.15.125:0/52546056 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f3994018d90 con 0x7f399806bee0 2023-10-12T02:13:43.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.510+0000 7f8f917fa640 1 -- 172.21.15.125:0/470810834 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f8f8800a300 con 0x7f8f9406d800 2023-10-12T02:13:43.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.671+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f71f0061d90 con 0x7f71f012d9f0 2023-10-12T02:13:43.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.671+0000 7f71e77fe640 1 -- 172.21.15.125:0/3571261763 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f71e000d150 con 0x7f71f012d9f0 2023-10-12T02:13:43.674 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.672+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9874061d90 con 0x7f9874130df0 2023-10-12T02:13:43.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.673+0000 7f9870ff9640 1 -- 172.21.15.125:0/1858974398 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f986400cea0 con 0x7f9874130df0 2023-10-12T02:13:43.675 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f71d0037890 msgr2=0x7f71d0039da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f71d0037890 0x7f71d0039da0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f012d9f0 msgr2=0x7f71f01c3440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f012d9f0 0x7f71f01c3440 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 shutdown_connections 2023-10-12T02:13:43.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f71d0037890 0x7f71d0039da0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 --2- 172.21.15.125:0/3571261763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71f012d9f0 0x7f71f01c3440 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 >> 172.21.15.125:0/3571261763 conn(0x7f71f009ff80 msgr2=0x7f71f009d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 shutdown_connections 2023-10-12T02:13:43.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f71f579e640 1 -- 172.21.15.125:0/3571261763 wait complete. 2023-10-12T02:13:43.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f9860037840 msgr2=0x7f9860039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f987a59e640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f9860037840 0x7f9860039d50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874130df0 msgr2=0x7f987411b640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.674+0000 7f987a59e640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874130df0 0x7f987411b640 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.675+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 shutdown_connections 2023-10-12T02:13:43.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.675+0000 7f987a59e640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f9860037840 0x7f9860039d50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.675+0000 7f987a59e640 1 --2- 172.21.15.125:0/1858974398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874130df0 0x7f987411b640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.675+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 >> 172.21.15.125:0/1858974398 conn(0x7f987409f950 msgr2=0x7f98740a1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.675+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 shutdown_connections 2023-10-12T02:13:43.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.675+0000 7f987a59e640 1 -- 172.21.15.125:0/1858974398 wait complete. 2023-10-12T02:13:43.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.683+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f39740052c0 con 0x7f399806bee0 2023-10-12T02:13:43.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.683+0000 7f398e7fc640 1 -- 172.21.15.125:0/52546056 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f3994018980 con 0x7f399806bee0 2023-10-12T02:13:43.686 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f397c037840 msgr2=0x7f397c039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f397c037840 0x7f397c039d50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399806bee0 msgr2=0x7f399811e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399806bee0 0x7f399811e000 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8f94061d90 con 0x7f8f9406d800 2023-10-12T02:13:43.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 shutdown_connections 2023-10-12T02:13:43.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f397c037840 0x7f397c039d50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 --2- 172.21.15.125:0/52546056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399806bee0 0x7f399811e000 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 >> 172.21.15.125:0/52546056 conn(0x7f399809f840 msgr2=0x7f3998117000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 shutdown_connections 2023-10-12T02:13:43.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f397bfff640 1 -- 172.21.15.125:0/52546056 wait complete. 2023-10-12T02:13:43.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.685+0000 7f8f917fa640 1 -- 172.21.15.125:0/470810834 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f8f8800ee40 con 0x7f8f9406d800 2023-10-12T02:13:43.691 INFO:teuthology.orchestra.run.smithi125.stdout:38654705670 2023-10-12T02:13:43.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f8f78037890 msgr2=0x7f8f78039da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f8f78037890 0x7f8f78039da0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9406d800 msgr2=0x7f8f94117000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9406d800 0x7f8f94117000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 shutdown_connections 2023-10-12T02:13:43.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f8f78037890 0x7f8f78039da0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 --2- 172.21.15.125:0/470810834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f9406d800 0x7f8f94117000 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 >> 172.21.15.125:0/470810834 conn(0x7f8f9409f840 msgr2=0x7f8f94129780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 shutdown_connections 2023-10-12T02:13:43.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.688+0000 7f8f9ae6d640 1 -- 172.21.15.125:0/470810834 wait complete. 2023-10-12T02:13:43.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2023-10-12T02:13:43.695 DEBUG:teuthology.parallel:result is None 2023-10-12T02:13:43.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2023-10-12T02:13:43.696 DEBUG:teuthology.parallel:result is None 2023-10-12T02:13:43.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2023-10-12T02:13:43.702 DEBUG:teuthology.parallel:result is None 2023-10-12T02:13:43.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.3 2023-10-12T02:13:43.704 DEBUG:teuthology.parallel:result is None 2023-10-12T02:13:43.705 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-12T02:13:43.709 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-12T02:13:43.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.842+0000 7fb9b9a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.846+0000 7fb9b9a6d640 1 Processor -- start 2023-10-12T02:13:43.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.846+0000 7fb9b9a6d640 1 -- start start 2023-10-12T02:13:43.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.846+0000 7fb9b9a6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fb9b412f5d0 0x7fb9b412f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.846+0000 7fb9b9a6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b406e400 con 0x7fb9b406bee0 2023-10-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.846+0000 7fb9b9a6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b409a630 con 0x7fb9b412f5d0 2023-10-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b37fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fb9b412f5d0 0x7fb9b412f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b37fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fb9b412f5d0 0x7fb9b412f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50038/0 (socket says 172.21.15.125:50038) 2023-10-12T02:13:43.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b37fe640 1 -- 172.21.15.125:0/1066547930 learned_addr learned my addr 172.21.15.125:0/1066547930 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b2ffd640 1 --1- 172.21.15.125:0/1066547930 >> v1:172.21.15.125:6789/0 conn(0x7fb9b406bee0 0x7fb9b406c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32914/0 (socket says 172.21.15.125:32914) 2023-10-12T02:13:43.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b37fe640 1 -- 172.21.15.125:0/1066547930 >> v1:172.21.15.125:6789/0 conn(0x7fb9b406bee0 legacy=0x7fb9b406c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b37fe640 1 -- 172.21.15.125:0/1066547930 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b406dcb0 con 0x7fb9b412f5d0 2023-10-12T02:13:43.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b37fe640 1 --2- 172.21.15.125:0/1066547930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b412f5d0 0x7fb9b412f9e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed797293cac4fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b27fc640 1 -- 172.21.15.125:0/1066547930 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb9a8010570 con 0x7fb9b412f5d0 2023-10-12T02:13:43.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b27fc640 1 -- 172.21.15.125:0/1066547930 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb9a80106d0 con 0x7fb9b412f5d0 2023-10-12T02:13:43.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.847+0000 7fb9b27fc640 1 -- 172.21.15.125:0/1066547930 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb9a8010970 con 0x7fb9b412f5d0 2023-10-12T02:13:43.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/1066547930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b412f5d0 msgr2=0x7fb9b412f9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:43.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 1 --2- 172.21.15.125:0/1066547930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b412f5d0 0x7fb9b412f9e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/1066547930 shutdown_connections 2023-10-12T02:13:43.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 1 --2- 172.21.15.125:0/1066547930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b412f5d0 0x7fb9b412f9e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:43.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/1066547930 >> 172.21.15.125:0/1066547930 conn(0x7fb9b409f840 msgr2=0x7fb9b409fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:43.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/1066547930 shutdown_connections 2023-10-12T02:13:43.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/1066547930 wait complete. 2023-10-12T02:13:43.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.848+0000 7fb9b9a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:43.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.849+0000 7fb9b9a6d640 1 Processor -- start 2023-10-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.849+0000 7fb9b9a6d640 1 -- start start 2023-10-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b9a6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b406bee0 0x7fb9b411dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b9a6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b412efb0 con 0x7fb9b406bee0 2023-10-12T02:13:43.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b406bee0 0x7fb9b411dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b406bee0 0x7fb9b411dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50054/0 (socket says 172.21.15.125:50054) 2023-10-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b37fe640 1 -- 172.21.15.125:0/4283925996 learned_addr learned my addr 172.21.15.125:0/4283925996 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b37fe640 1 -- 172.21.15.125:0/4283925996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b411d860 con 0x7fb9b406bee0 2023-10-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b37fe640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b406bee0 0x7fb9b411dc00 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb99bfff640 1 -- 172.21.15.125:0/4283925996 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb9a800fa60 con 0x7fb9b406bee0 2023-10-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb99bfff640 1 -- 172.21.15.125:0/4283925996 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb9a8016040 con 0x7fb9b406bee0 2023-10-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b4121590 con 0x7fb9b406bee0 2023-10-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb99bfff640 1 -- 172.21.15.125:0/4283925996 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb9a800fcf0 con 0x7fb9b406bee0 2023-10-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.850+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b411d4b0 con 0x7fb9b406bee0 2023-10-12T02:13:43.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.851+0000 7fb99bfff640 1 -- 172.21.15.125:0/4283925996 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7fb9a800e360 con 0x7fb9b406bee0 2023-10-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.851+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b411faa0 con 0x7fb9b406bee0 2023-10-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.851+0000 7fb99bfff640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fb9a0037840 0x7fb9a0039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.851+0000 7fb99bfff640 1 -- 172.21.15.125:0/4283925996 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb9a804ab90 con 0x7fb9b406bee0 2023-10-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.852+0000 7fb9b2ffd640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fb9a0037840 0x7fb9a0039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:43.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.852+0000 7fb9b2ffd640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fb9a0037840 0x7fb9a0039d50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:43.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:43.856+0000 7fb99bfff640 1 -- 172.21.15.125:0/4283925996 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7fb9a8018c60 con 0x7fb9b406bee0 2023-10-12T02:13:44.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.021+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 --> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9b40a05c0 con 0x7fb9a0037840 2023-10-12T02:13:44.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.023+0000 7fb99bfff640 1 -- 172.21.15.125:0/4283925996 <== mgr.4099 v2:172.21.15.158:6800/1559993426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+30046 (crc 0 0 0) 0x7fb9a40070b0 con 0x7fb9a0037840 2023-10-12T02:13:44.025 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:44.026 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-12T02:13:44.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fb9a0037840 msgr2=0x7fb9a0039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fb9a0037840 0x7fb9a0039d50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b406bee0 msgr2=0x7fb9b411dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b406bee0 0x7fb9b411dc00 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 shutdown_connections 2023-10-12T02:13:44.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7fb9a0037840 0x7fb9a0039d50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 --2- 172.21.15.125:0/4283925996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9b406bee0 0x7fb9b411dc00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 >> 172.21.15.125:0/4283925996 conn(0x7fb9b409f840 msgr2=0x7fb9b4117000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:44.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 shutdown_connections 2023-10-12T02:13:44.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.025+0000 7fb9b9a6d640 1 -- 172.21.15.125:0/4283925996 wait complete. 2023-10-12T02:13:44.037 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-10-12T02:13:43.251771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109572,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320828,"statfs":{"total":429496729600,"available":429384527872,"internally_reserved":0,"allocated":1421312,"data_stored":1261588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.221513+0000","last_change":"2023-10-12T02:13:42.221842+0000","last_active":"2023-10-12T02:13:42.221513+0000","last_peered":"2023-10-12T02:13:42.221513+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.219091+0000","last_became_peered":"2023-10-12T02:13:42.219091+0000","last_unstale":"2023-10-12T02:13:42.221513+0000","last_undegraded":"2023-10-12T02:13:42.221513+0000","last_fullsized":"2023-10-12T02:13:42.221513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.218185+0000","last_change":"2023-10-12T02:13:42.218566+0000","last_active":"2023-10-12T02:13:42.218185+0000","last_peered":"2023-10-12T02:13:42.218185+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.215146+0000","last_became_peered":"2023-10-12T02:13:42.215146+0000","last_unstale":"2023-10-12T02:13:42.218185+0000","last_undegraded":"2023-10-12T02:13:42.218185+0000","last_fullsized":"2023-10-12T02:13:42.218185+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-12T02:13:43.162800+0000","last_change":"2023-10-12T02:13:42.223707+0000","last_active":"2023-10-12T02:13:43.162800+0000","last_peered":"2023-10-12T02:13:43.162800+0000","last_clean":"2023-10-12T02:13:43.162800+0000","last_became_active":"2023-10-12T02:13:42.219739+0000","last_became_peered":"2023-10-12T02:13:42.219739+0000","last_unstale":"2023-10-12T02:13:43.162800+0000","last_undegraded":"2023-10-12T02:13:43.162800+0000","last_fullsized":"2023-10-12T02:13:43.162800+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.211573+0000","last_change":"2023-10-12T02:13:42.212011+0000","last_active":"2023-10-12T02:13:42.211573+0000","last_peered":"2023-10-12T02:13:42.211573+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.208030+0000","last_became_peered":"2023-10-12T02:13:42.208030+0000","last_unstale":"2023-10-12T02:13:42.211573+0000","last_undegraded":"2023-10-12T02:13:42.211573+0000","last_fullsized":"2023-10-12T02:13:42.211573+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.216744+0000","last_change":"2023-10-12T02:13:42.217172+0000","last_active":"2023-10-12T02:13:42.216744+0000","last_peered":"2023-10-12T02:13:42.216744+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.214289+0000","last_became_peered":"2023-10-12T02:13:42.214289+0000","last_unstale":"2023-10-12T02:13:42.216744+0000","last_undegraded":"2023-10-12T02:13:42.216744+0000","last_fullsized":"2023-10-12T02:13:42.216744+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.201581+0000","last_change":"2023-10-12T02:13:42.202008+0000","last_active":"2023-10-12T02:13:42.201581+0000","last_peered":"2023-10-12T02:13:42.201581+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.196808+0000","last_became_peered":"2023-10-12T02:13:42.196808+0000","last_unstale":"2023-10-12T02:13:42.201581+0000","last_undegraded":"2023-10-12T02:13:42.201581+0000","last_fullsized":"2023-10-12T02:13:42.201581+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.205032+0000","last_change":"2023-10-12T02:13:42.205411+0000","last_active":"2023-10-12T02:13:42.205032+0000","last_peered":"2023-10-12T02:13:42.205032+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.202370+0000","last_became_peered":"2023-10-12T02:13:42.202370+0000","last_unstale":"2023-10-12T02:13:42.205032+0000","last_undegraded":"2023-10-12T02:13:42.205032+0000","last_fullsized":"2023-10-12T02:13:42.205032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.218074+0000","last_change":"2023-10-12T02:13:42.218418+0000","last_active":"2023-10-12T02:13:42.218074+0000","last_peered":"2023-10-12T02:13:42.218074+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.215229+0000","last_became_peered":"2023-10-12T02:13:42.215229+0000","last_unstale":"2023-10-12T02:13:42.218074+0000","last_undegraded":"2023-10-12T02:13:42.218074+0000","last_fullsized":"2023-10-12T02:13:42.218074+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":141,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-12T02:13:43.164134+0000","last_change":"2023-10-12T02:13:40.170078+0000","last_active":"2023-10-12T02:13:43.164134+0000","last_peered":"2023-10-12T02:13:43.164134+0000","last_clean":"2023-10-12T02:13:43.164134+0000","last_became_active":"2023-10-12T02:13:40.166588+0000","last_became_peered":"2023-10-12T02:13:40.166588+0000","last_unstale":"2023-10-12T02:13:43.164134+0000","last_undegraded":"2023-10-12T02:13:43.164134+0000","last_fullsized":"2023-10-12T02:13:43.164134+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:39.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:39.132236+0000","last_clean_scrub_stamp":"2023-10-12T02:13:39.132236+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27780,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829820,"statfs":{"total":107374182400,"available":107345735680,"internally_reserved":0,"allocated":655360,"data_stored":612552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27008,"kb_used_data":56,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830592,"statfs":{"total":107374182400,"available":107346526208,"internally_reserved":0,"allocated":57344,"data_stored":19556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27008,"kb_used_data":56,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830592,"statfs":{"total":107374182400,"available":107346526208,"internally_reserved":0,"allocated":57344,"data_stored":19556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27776,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829824,"statfs":{"total":107374182400,"available":107345739776,"internally_reserved":0,"allocated":651264,"data_stored":609924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-10-12T02:13:44.038 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-12T02:13:44.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.140+0000 7f69f879e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:44.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.143+0000 7f69f879e640 1 Processor -- start 2023-10-12T02:13:44.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.143+0000 7f69f879e640 1 -- start start 2023-10-12T02:13:44.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f879e640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f69f0130720 0x7f69f0134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:44.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f879e640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f009bc60 con 0x7f69f006d250 2023-10-12T02:13:44.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f879e640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f009a000 con 0x7f69f0130720 2023-10-12T02:13:44.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f5d39640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f69f0130720 0x7f69f0134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:44.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f653a640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f69f006d250 0x7f69f0130010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32928/0 (socket says 172.21.15.125:32928) 2023-10-12T02:13:44.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f653a640 1 -- 172.21.15.125:0/1598636439 learned_addr learned my addr 172.21.15.125:0/1598636439 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:44.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f5d39640 1 -- 172.21.15.125:0/1598636439 >> v1:172.21.15.125:6789/0 conn(0x7f69f006d250 legacy=0x7f69f0130010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f5d39640 1 -- 172.21.15.125:0/1598636439 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f009b4c0 con 0x7f69f0130720 2023-10-12T02:13:44.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f5d39640 1 --2- 172.21.15.125:0/1598636439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f0130720 0x7f69f0134dc0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d593a9604c7074c server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:44.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.144+0000 7f69f5538640 1 -- 172.21.15.125:0/1598636439 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69e0009ce0 con 0x7f69f0130720 2023-10-12T02:13:44.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f5538640 1 -- 172.21.15.125:0/1598636439 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f69e0009e40 con 0x7f69f0130720 2023-10-12T02:13:44.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f5538640 1 -- 172.21.15.125:0/1598636439 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69e000e360 con 0x7f69f0130720 2023-10-12T02:13:44.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f879e640 1 -- 172.21.15.125:0/1598636439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f0130720 msgr2=0x7f69f0134dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f879e640 1 --2- 172.21.15.125:0/1598636439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f0130720 0x7f69f0134dc0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f879e640 1 -- 172.21.15.125:0/1598636439 shutdown_connections 2023-10-12T02:13:44.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f879e640 1 --2- 172.21.15.125:0/1598636439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f0130720 0x7f69f0134dc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f879e640 1 -- 172.21.15.125:0/1598636439 >> 172.21.15.125:0/1598636439 conn(0x7f69f009fad0 msgr2=0x7f69f009fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:44.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f879e640 1 -- 172.21.15.125:0/1598636439 shutdown_connections 2023-10-12T02:13:44.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.145+0000 7f69f879e640 1 -- 172.21.15.125:0/1598636439 wait complete. 2023-10-12T02:13:44.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.146+0000 7f69f879e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:44.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.146+0000 7f69f879e640 1 Processor -- start 2023-10-12T02:13:44.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.146+0000 7f69f879e640 1 -- start start 2023-10-12T02:13:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.147+0000 7f69f879e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f006d250 0x7f69f01c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.147+0000 7f69f879e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f01c5500 con 0x7f69f006d250 2023-10-12T02:13:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.147+0000 7f69f653a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f006d250 0x7f69f01c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.147+0000 7f69f653a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f006d250 0x7f69f01c5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50066/0 (socket says 172.21.15.125:50066) 2023-10-12T02:13:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.147+0000 7f69f653a640 1 -- 172.21.15.125:0/636728076 learned_addr learned my addr 172.21.15.125:0/636728076 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:44.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.147+0000 7f69f653a640 1 -- 172.21.15.125:0/636728076 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0071620 con 0x7f69f006d250 2023-10-12T02:13:44.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.148+0000 7f69f653a640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f006d250 0x7f69f01c5a50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:44.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.148+0000 7f69e6ffd640 1 -- 172.21.15.125:0/636728076 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69ec0070b0 con 0x7f69f006d250 2023-10-12T02:13:44.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.148+0000 7f69e6ffd640 1 -- 172.21.15.125:0/636728076 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f69ec007210 con 0x7f69f006d250 2023-10-12T02:13:44.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.148+0000 7f69e6ffd640 1 -- 172.21.15.125:0/636728076 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69ec0074b0 con 0x7f69f006d250 2023-10-12T02:13:44.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.148+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f0118de0 con 0x7f69f006d250 2023-10-12T02:13:44.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.148+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f01184e0 con 0x7f69f006d250 2023-10-12T02:13:44.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.149+0000 7f69e6ffd640 1 -- 172.21.15.125:0/636728076 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f69ec0076a0 con 0x7f69f006d250 2023-10-12T02:13:44.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.149+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f0071a60 con 0x7f69f006d250 2023-10-12T02:13:44.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.149+0000 7f69e6ffd640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f69d4037840 0x7f69d4039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:44.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.149+0000 7f69e6ffd640 1 -- 172.21.15.125:0/636728076 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f69ec047790 con 0x7f69f006d250 2023-10-12T02:13:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.149+0000 7f69f5d39640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f69d4037840 0x7f69d4039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.150+0000 7f69f5d39640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f69d4037840 0x7f69d4039d50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.154+0000 7f69e6ffd640 1 -- 172.21.15.125:0/636728076 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f69ec0090b0 con 0x7f69f006d250 2023-10-12T02:13:44.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.316+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 --> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69f009af00 con 0x7f69d4037840 2023-10-12T02:13:44.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.317+0000 7f69e6ffd640 1 -- 172.21.15.125:0/636728076 <== mgr.4099 v2:172.21.15.158:6800/1559993426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+30046 (crc 0 0 0) 0x7f69e00124e0 con 0x7f69d4037840 2023-10-12T02:13:44.320 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:44.321 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-12T02:13:44.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f69d4037840 msgr2=0x7f69d4039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f69d4037840 0x7f69d4039d50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f006d250 msgr2=0x7f69f01c5a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f006d250 0x7f69f01c5a50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 shutdown_connections 2023-10-12T02:13:44.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f69d4037840 0x7f69d4039d50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 --2- 172.21.15.125:0/636728076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69f006d250 0x7f69f01c5a50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 >> 172.21.15.125:0/636728076 conn(0x7f69f009fad0 msgr2=0x7f69f01b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:44.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 shutdown_connections 2023-10-12T02:13:44.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.319+0000 7f69f879e640 1 -- 172.21.15.125:0/636728076 wait complete. 2023-10-12T02:13:44.331 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-10-12T02:13:43.251771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109572,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320828,"statfs":{"total":429496729600,"available":429384527872,"internally_reserved":0,"allocated":1421312,"data_stored":1261588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.221513+0000","last_change":"2023-10-12T02:13:42.221842+0000","last_active":"2023-10-12T02:13:42.221513+0000","last_peered":"2023-10-12T02:13:42.221513+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.219091+0000","last_became_peered":"2023-10-12T02:13:42.219091+0000","last_unstale":"2023-10-12T02:13:42.221513+0000","last_undegraded":"2023-10-12T02:13:42.221513+0000","last_fullsized":"2023-10-12T02:13:42.221513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.218185+0000","last_change":"2023-10-12T02:13:42.218566+0000","last_active":"2023-10-12T02:13:42.218185+0000","last_peered":"2023-10-12T02:13:42.218185+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.215146+0000","last_became_peered":"2023-10-12T02:13:42.215146+0000","last_unstale":"2023-10-12T02:13:42.218185+0000","last_undegraded":"2023-10-12T02:13:42.218185+0000","last_fullsized":"2023-10-12T02:13:42.218185+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-12T02:13:43.162800+0000","last_change":"2023-10-12T02:13:42.223707+0000","last_active":"2023-10-12T02:13:43.162800+0000","last_peered":"2023-10-12T02:13:43.162800+0000","last_clean":"2023-10-12T02:13:43.162800+0000","last_became_active":"2023-10-12T02:13:42.219739+0000","last_became_peered":"2023-10-12T02:13:42.219739+0000","last_unstale":"2023-10-12T02:13:43.162800+0000","last_undegraded":"2023-10-12T02:13:43.162800+0000","last_fullsized":"2023-10-12T02:13:43.162800+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.211573+0000","last_change":"2023-10-12T02:13:42.212011+0000","last_active":"2023-10-12T02:13:42.211573+0000","last_peered":"2023-10-12T02:13:42.211573+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.208030+0000","last_became_peered":"2023-10-12T02:13:42.208030+0000","last_unstale":"2023-10-12T02:13:42.211573+0000","last_undegraded":"2023-10-12T02:13:42.211573+0000","last_fullsized":"2023-10-12T02:13:42.211573+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.216744+0000","last_change":"2023-10-12T02:13:42.217172+0000","last_active":"2023-10-12T02:13:42.216744+0000","last_peered":"2023-10-12T02:13:42.216744+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.214289+0000","last_became_peered":"2023-10-12T02:13:42.214289+0000","last_unstale":"2023-10-12T02:13:42.216744+0000","last_undegraded":"2023-10-12T02:13:42.216744+0000","last_fullsized":"2023-10-12T02:13:42.216744+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.201581+0000","last_change":"2023-10-12T02:13:42.202008+0000","last_active":"2023-10-12T02:13:42.201581+0000","last_peered":"2023-10-12T02:13:42.201581+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.196808+0000","last_became_peered":"2023-10-12T02:13:42.196808+0000","last_unstale":"2023-10-12T02:13:42.201581+0000","last_undegraded":"2023-10-12T02:13:42.201581+0000","last_fullsized":"2023-10-12T02:13:42.201581+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.205032+0000","last_change":"2023-10-12T02:13:42.205411+0000","last_active":"2023-10-12T02:13:42.205032+0000","last_peered":"2023-10-12T02:13:42.205032+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.202370+0000","last_became_peered":"2023-10-12T02:13:42.202370+0000","last_unstale":"2023-10-12T02:13:42.205032+0000","last_undegraded":"2023-10-12T02:13:42.205032+0000","last_fullsized":"2023-10-12T02:13:42.205032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-12T02:13:42.218074+0000","last_change":"2023-10-12T02:13:42.218418+0000","last_active":"2023-10-12T02:13:42.218074+0000","last_peered":"2023-10-12T02:13:42.218074+0000","last_clean":"2023-10-12T02:13:41.140600+0000","last_became_active":"2023-10-12T02:13:42.215229+0000","last_became_peered":"2023-10-12T02:13:42.215229+0000","last_unstale":"2023-10-12T02:13:42.218074+0000","last_undegraded":"2023-10-12T02:13:42.218074+0000","last_fullsized":"2023-10-12T02:13:42.218074+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:41.140600+0000","last_clean_scrub_stamp":"2023-10-12T02:13:41.140600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":141,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-12T02:13:43.164134+0000","last_change":"2023-10-12T02:13:40.170078+0000","last_active":"2023-10-12T02:13:43.164134+0000","last_peered":"2023-10-12T02:13:43.164134+0000","last_clean":"2023-10-12T02:13:43.164134+0000","last_became_active":"2023-10-12T02:13:40.166588+0000","last_became_peered":"2023-10-12T02:13:40.166588+0000","last_unstale":"2023-10-12T02:13:43.164134+0000","last_undegraded":"2023-10-12T02:13:43.164134+0000","last_fullsized":"2023-10-12T02:13:43.164134+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-12T02:13:39.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-12T02:13:39.132236+0000","last_clean_scrub_stamp":"2023-10-12T02:13:39.132236+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27780,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829820,"statfs":{"total":107374182400,"available":107345735680,"internally_reserved":0,"allocated":655360,"data_stored":612552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27008,"kb_used_data":56,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830592,"statfs":{"total":107374182400,"available":107346526208,"internally_reserved":0,"allocated":57344,"data_stored":19556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27008,"kb_used_data":56,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830592,"statfs":{"total":107374182400,"available":107346526208,"internally_reserved":0,"allocated":57344,"data_stored":19556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27776,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829824,"statfs":{"total":107374182400,"available":107345739776,"internally_reserved":0,"allocated":651264,"data_stored":609924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-10-12T02:13:44.333 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-12T02:13:44.333 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-12T02:13:44.333 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-12T02:13:44.334 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-12T02:13:44.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.434+0000 7f747fa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:44.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.436+0000 7f747fa6d640 1 Processor -- start 2023-10-12T02:13:44.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.436+0000 7f747fa6d640 1 -- start start 2023-10-12T02:13:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747fa6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f74780701d0 0x7f7478136540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747fa6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74780723e0 con 0x7f747806f6b0 2023-10-12T02:13:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747fa6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74780713b0 con 0x7f74780701d0 2023-10-12T02:13:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747d008640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f74780701d0 0x7f7478136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747d809640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f747806f6b0 0x7f747806fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:32944/0 (socket says 172.21.15.125:32944) 2023-10-12T02:13:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747d008640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f74780701d0 0x7f7478136540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50074/0 (socket says 172.21.15.125:50074) 2023-10-12T02:13:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747d809640 1 -- 172.21.15.125:0/2826315337 learned_addr learned my addr 172.21.15.125:0/2826315337 (peer_addr_for_me v1:172.21.15.125:0/0) 2023-10-12T02:13:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747d008640 1 -- 172.21.15.125:0/2826315337 >> v1:172.21.15.125:6789/0 conn(0x7f747806f6b0 legacy=0x7f747806fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.437+0000 7f747d008640 1 -- 172.21.15.125:0/2826315337 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780710a0 con 0x7f74780701d0 2023-10-12T02:13:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747d008640 1 --2- 172.21.15.125:0/2826315337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74780701d0 0x7f7478136540 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d546f26df493c90e server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747c807640 1 -- 172.21.15.125:0/2826315337 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7468009ce0 con 0x7f74780701d0 2023-10-12T02:13:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747c807640 1 -- 172.21.15.125:0/2826315337 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7468009e40 con 0x7f74780701d0 2023-10-12T02:13:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747c807640 1 -- 172.21.15.125:0/2826315337 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f746800e360 con 0x7f74780701d0 2023-10-12T02:13:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747fa6d640 1 -- 172.21.15.125:0/2826315337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74780701d0 msgr2=0x7f7478136540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747fa6d640 1 --2- 172.21.15.125:0/2826315337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74780701d0 0x7f7478136540 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747fa6d640 1 -- 172.21.15.125:0/2826315337 shutdown_connections 2023-10-12T02:13:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747fa6d640 1 --2- 172.21.15.125:0/2826315337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74780701d0 0x7f7478136540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.438+0000 7f747fa6d640 1 -- 172.21.15.125:0/2826315337 >> 172.21.15.125:0/2826315337 conn(0x7f74780a14d0 msgr2=0x7f74780a18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.439+0000 7f747fa6d640 1 -- 172.21.15.125:0/2826315337 shutdown_connections 2023-10-12T02:13:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.439+0000 7f747fa6d640 1 -- 172.21.15.125:0/2826315337 wait complete. 2023-10-12T02:13:44.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.439+0000 7f747fa6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:13:44.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.440+0000 7f747fa6d640 1 Processor -- start 2023-10-12T02:13:44.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.440+0000 7f747fa6d640 1 -- start start 2023-10-12T02:13:44.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.440+0000 7f747fa6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f747806f6b0 0x7f7478097360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:44.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.440+0000 7f747fa6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7478096d20 con 0x7f747806f6b0 2023-10-12T02:13:44.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.440+0000 7f747d809640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f747806f6b0 0x7f7478097360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:44.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.440+0000 7f747d809640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f747806f6b0 0x7f7478097360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50086/0 (socket says 172.21.15.125:50086) 2023-10-12T02:13:44.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.440+0000 7f747d809640 1 -- 172.21.15.125:0/1228001909 learned_addr learned my addr 172.21.15.125:0/1228001909 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:13:44.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.441+0000 7f747d809640 1 -- 172.21.15.125:0/1228001909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478096fc0 con 0x7f747806f6b0 2023-10-12T02:13:44.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.441+0000 7f747d809640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f747806f6b0 0x7f7478097360 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:44.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.441+0000 7f746e7fc640 1 -- 172.21.15.125:0/1228001909 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7474005c20 con 0x7f747806f6b0 2023-10-12T02:13:44.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.441+0000 7f746e7fc640 1 -- 172.21.15.125:0/1228001909 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7474005d80 con 0x7f747806f6b0 2023-10-12T02:13:44.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.441+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7478096a10 con 0x7f747806f6b0 2023-10-12T02:13:44.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.441+0000 7f746e7fc640 1 -- 172.21.15.125:0/1228001909 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f747400cbd0 con 0x7f747806f6b0 2023-10-12T02:13:44.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.441+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7478095dd0 con 0x7f747806f6b0 2023-10-12T02:13:44.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.442+0000 7f746e7fc640 1 -- 172.21.15.125:0/1228001909 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f74740070b0 con 0x7f747806f6b0 2023-10-12T02:13:44.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.442+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74780962b0 con 0x7f747806f6b0 2023-10-12T02:13:44.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.442+0000 7f746e7fc640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f745c037840 0x7f745c039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:13:44.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.442+0000 7f746e7fc640 1 -- 172.21.15.125:0/1228001909 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7474047070 con 0x7f747806f6b0 2023-10-12T02:13:44.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.443+0000 7f747d008640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f745c037840 0x7f745c039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:13:44.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.443+0000 7f747d008640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f745c037840 0x7f745c039d50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:13:44.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.447+0000 7f746e7fc640 1 -- 172.21.15.125:0/1228001909 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (crc 0 0 0) 0x7f747400dbb0 con 0x7f747806f6b0 2023-10-12T02:13:44.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.646+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f74780971b0 con 0x7f747806f6b0 2023-10-12T02:13:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.646+0000 7f746e7fc640 1 -- 172.21.15.125:0/1228001909 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (crc 0 0 0) 0x7f747400d150 con 0x7f747806f6b0 2023-10-12T02:13:44.649 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-12T02:13:44.650 INFO:teuthology.orchestra.run.smithi125.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-12T02:13:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f745c037840 msgr2=0x7f745c039d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f745c037840 0x7f745c039d50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f747806f6b0 msgr2=0x7f7478097360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:13:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f747806f6b0 0x7f7478097360 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 shutdown_connections 2023-10-12T02:13:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f745c037840 0x7f745c039d50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 --2- 172.21.15.125:0/1228001909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f747806f6b0 0x7f7478097360 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:13:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 >> 172.21.15.125:0/1228001909 conn(0x7f74780a14d0 msgr2=0x7f747808bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:13:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 shutdown_connections 2023-10-12T02:13:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T02:13:44.648+0000 7f747fa6d640 1 -- 172.21.15.125:0/1228001909 wait complete. 2023-10-12T02:13:44.660 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-12T02:13:44.661 INFO:teuthology.run_tasks:Running task workunit... 2023-10-12T02:13:44.675 INFO:tasks.workunit:Pulling workunits from ref 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 2023-10-12T02:13:44.676 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-12T02:13:44.676 DEBUG:teuthology.orchestra.run.smithi125:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-12T02:13:44.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T02:13:44.691 INFO:teuthology.orchestra.run.smithi125.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-12T02:13:44.692 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-10-12T02:13:44.746 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-10-12T02:13:44.747 DEBUG:teuthology.orchestra.run.smithi125:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-10-12T02:13:44.803 INFO:tasks.workunit:timeout=3h 2023-10-12T02:13:44.803 INFO:tasks.workunit:cleanup=True 2023-10-12T02:13:44.803 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 2023-10-12T02:13:44.857 INFO:tasks.workunit.client.0.smithi125.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-12T02:14:46.876 INFO:tasks.workunit.client.0.smithi125.stderr:Note: switching to '9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5'. 2023-10-12T02:14:46.877 INFO:tasks.workunit.client.0.smithi125.stderr: 2023-10-12T02:14:46.877 INFO:tasks.workunit.client.0.smithi125.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-12T02:14:46.877 INFO:tasks.workunit.client.0.smithi125.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-12T02:14:46.877 INFO:tasks.workunit.client.0.smithi125.stderr:state without impacting any branches by switching back to a branch. 2023-10-12T02:14:46.878 INFO:tasks.workunit.client.0.smithi125.stderr: 2023-10-12T02:14:46.878 INFO:tasks.workunit.client.0.smithi125.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-12T02:14:46.878 INFO:tasks.workunit.client.0.smithi125.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-12T02:14:46.878 INFO:tasks.workunit.client.0.smithi125.stderr: 2023-10-12T02:14:46.878 INFO:tasks.workunit.client.0.smithi125.stderr: git switch -c 2023-10-12T02:14:46.879 INFO:tasks.workunit.client.0.smithi125.stderr: 2023-10-12T02:14:46.879 INFO:tasks.workunit.client.0.smithi125.stderr:Or undo this operation with: 2023-10-12T02:14:46.879 INFO:tasks.workunit.client.0.smithi125.stderr: 2023-10-12T02:14:46.879 INFO:tasks.workunit.client.0.smithi125.stderr: git switch - 2023-10-12T02:14:46.879 INFO:tasks.workunit.client.0.smithi125.stderr: 2023-10-12T02:14:46.880 INFO:tasks.workunit.client.0.smithi125.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-12T02:14:46.880 INFO:tasks.workunit.client.0.smithi125.stderr: 2023-10-12T02:14:46.880 INFO:tasks.workunit.client.0.smithi125.stderr:HEAD is now at 9587ffdfc8c crimson/common/operation: drop cross-core supports in ExitBarrier 2023-10-12T02:14:46.881 DEBUG:teuthology.orchestra.run.smithi125:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-12T02:14:46.949 INFO:tasks.workunit.client.0.smithi125.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-12T02:14:46.951 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-12T02:14:46.951 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-12T02:14:47.809 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-12T02:14:47.874 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-12T02:14:47.916 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-12T02:14:47.918 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-12T02:14:47.918 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-12T02:14:47.958 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-12T02:14:47.961 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-12T02:14:47.961 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-12T02:14:48.016 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2023-10-12T02:14:48.017 INFO:tasks.workunit:Running workunit rados/test.sh... 2023-10-12T02:14:48.017 DEBUG:teuthology.orchestra.run.smithi125:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2023-10-12T02:14:48.075 INFO:tasks.workunit.client.0.smithi125.stderr:+ parallel=1 2023-10-12T02:14:48.075 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' '' = --serial ']' 2023-10-12T02:14:48.076 INFO:tasks.workunit.client.0.smithi125.stderr:+ color= 2023-10-12T02:14:48.076 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -t 1 ']' 2023-10-12T02:14:48.076 INFO:tasks.workunit.client.0.smithi125.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2023-10-12T02:14:48.076 INFO:tasks.workunit.client.0.smithi125.stderr:+ declare -A pids 2023-10-12T02:14:48.076 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.077 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.077 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_aio 2023-10-12T02:14:48.077 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_aio' 2023-10-12T02:14:48.077 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_aio 2023-10-12T02:14:48.077 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.078 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_aio 2023-10-12T02:14:48.078 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83561 2023-10-12T02:14:48.079 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_aio on pid 83561' 2023-10-12T02:14:48.079 INFO:tasks.workunit.client.0.smithi125.stdout:test api_aio on pid 83561 2023-10-12T02:14:48.082 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83561 2023-10-12T02:14:48.082 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2023-10-12T02:14:48.083 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.083 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.083 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_aio_pp 2023-10-12T02:14:48.083 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_aio_pp' 2023-10-12T02:14:48.083 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_aio_pp 2023-10-12T02:14:48.084 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.084 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.084 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.084 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.084 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.085 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.085 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.085 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.085 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.086 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.086 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.086 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.086 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.086 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.087 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.087 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.087 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.087 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.087 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.088 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.088 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.088 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.088 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.088 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.089 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.089 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.089 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_aio_pp 2023-10-12T02:14:48.089 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83568 2023-10-12T02:14:48.090 INFO:tasks.workunit.client.0.smithi125.stdout:test api_aio_pp on pid 83568 2023-10-12T02:14:48.093 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_aio_pp on pid 83568' 2023-10-12T02:14:48.093 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83568 2023-10-12T02:14:48.094 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.094 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.094 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2023-10-12T02:14:48.094 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_io 2023-10-12T02:14:48.094 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_io' 2023-10-12T02:14:48.095 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_io 2023-10-12T02:14:48.095 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.095 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.095 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.095 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.095 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.096 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.096 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.096 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.096 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.096 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.097 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.097 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.097 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.097 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.097 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.098 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.098 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.098 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.098 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.098 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.099 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.100 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.100 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.100 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.100 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.100 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.101 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.101 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.101 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.101 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.101 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.102 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.102 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.102 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.102 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.103 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.103 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.103 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.103 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.105 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.105 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.105 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.106 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_io 2023-10-12T02:14:48.106 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83577 2023-10-12T02:14:48.106 INFO:tasks.workunit.client.0.smithi125.stdout:test api_io on pid 83577 2023-10-12T02:14:48.110 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_io on pid 83577' 2023-10-12T02:14:48.110 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83577 2023-10-12T02:14:48.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2023-10-12T02:14:48.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.111 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_io_pp 2023-10-12T02:14:48.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_io_pp' 2023-10-12T02:14:48.112 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_io_pp 2023-10-12T02:14:48.112 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.112 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.112 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.112 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.112 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.113 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.113 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.113 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.113 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.113 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.114 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.114 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.114 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.114 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.114 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.115 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.115 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.115 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.115 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.115 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.115 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.116 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.116 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.116 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.116 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.116 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.117 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.117 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.117 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.117 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.117 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.118 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.118 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.118 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.118 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.118 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.119 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.119 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.119 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.119 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.119 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.119 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.121 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.121 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.121 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.121 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.121 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.121 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.122 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.122 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.122 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_io_pp 2023-10-12T02:14:48.122 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83588 2023-10-12T02:14:48.123 INFO:tasks.workunit.client.0.smithi125.stdout:test api_io_pp on pid 83588 2023-10-12T02:14:48.129 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_io_pp on pid 83588' 2023-10-12T02:14:48.129 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83588 2023-10-12T02:14:48.129 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.129 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.130 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2023-10-12T02:14:48.130 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_asio 2023-10-12T02:14:48.130 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_asio' 2023-10-12T02:14:48.130 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_asio 2023-10-12T02:14:48.130 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.131 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.131 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.131 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.131 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.131 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.132 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.132 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.132 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.132 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.132 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.132 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.133 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.133 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.133 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.133 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.133 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.134 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.134 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.134 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.134 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.134 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.135 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.135 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.135 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.135 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.135 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.136 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.136 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.136 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.136 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.136 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.137 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.137 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.137 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.137 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.137 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.137 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.138 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.138 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.138 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.138 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.138 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.139 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.139 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.139 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.139 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.139 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.140 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.140 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.140 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.140 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.140 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.141 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.141 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.141 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.141 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.141 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.142 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.142 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.142 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.142 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.142 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.143 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.143 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.143 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.143 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.143 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.144 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.144 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.144 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.144 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.144 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.145 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.145 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.145 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.145 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.145 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_asio 2023-10-12T02:14:48.145 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.146 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83603 2023-10-12T02:14:48.146 INFO:tasks.workunit.client.0.smithi125.stdout:test api_asio on pid 83603 2023-10-12T02:14:48.152 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_asio on pid 83603' 2023-10-12T02:14:48.153 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83603 2023-10-12T02:14:48.153 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.153 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.153 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2023-10-12T02:14:48.153 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.154 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_list 2023-10-12T02:14:48.154 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_list' 2023-10-12T02:14:48.154 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_list 2023-10-12T02:14:48.154 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.154 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.155 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.155 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.155 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.155 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.155 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.156 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.156 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.156 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.156 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.156 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.157 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.157 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.157 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.157 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.157 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.158 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.158 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.158 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.158 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.158 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.159 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.159 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.159 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.159 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.159 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.159 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.160 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.160 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.160 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.160 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.160 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.161 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.161 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.161 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.161 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.161 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.162 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.162 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.162 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.162 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.162 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.163 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.163 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.163 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.163 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.163 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.164 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.164 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.164 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.164 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.164 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.164 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.165 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.165 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.165 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.165 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.165 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.166 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.166 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.166 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.166 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.166 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.167 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.167 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.167 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.167 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.167 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.168 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.168 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.168 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.168 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.168 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.169 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.169 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.169 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.169 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.169 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.169 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.170 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.170 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.170 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.170 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.170 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.172 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.172 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.172 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.172 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.172 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83561/exe 2023-10-12T02:14:48.173 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_list 2023-10-12T02:14:48.173 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.173 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.173 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83620 2023-10-12T02:14:48.174 INFO:tasks.workunit.client.0.smithi125.stdout:test api_list on pid 83620 2023-10-12T02:14:48.180 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_list on pid 83620' 2023-10-12T02:14:48.181 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83620 2023-10-12T02:14:48.181 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.181 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.181 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.181 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.182 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2023-10-12T02:14:48.182 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.182 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.182 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_lock 2023-10-12T02:14:48.182 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.183 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_lock' 2023-10-12T02:14:48.183 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_lock 2023-10-12T02:14:48.183 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.183 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.183 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.184 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.184 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.184 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.184 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.184 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.185 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.185 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.185 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.185 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.185 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.186 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.186 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.186 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.186 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.186 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.187 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.187 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.187 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.187 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.187 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.187 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.188 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.188 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.188 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.188 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.188 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.189 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.189 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.189 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.189 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.189 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_aio 2023-10-12T02:14:48.190 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.190 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.190 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.190 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.190 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.191 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.191 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.191 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.191 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.191 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.192 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.192 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.192 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.192 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.192 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.192 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.193 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.193 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.193 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.193 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.193 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.194 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.194 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.194 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.194 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.194 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.195 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.195 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.195 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.195 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.195 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_aio: /' 2023-10-12T02:14:48.196 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.196 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.196 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.196 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.196 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.196 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.197 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.197 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.197 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.197 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.197 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.198 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.198 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.198 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.198 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.198 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.199 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.199 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.199 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.199 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.199 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.200 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.200 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.200 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.200 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.200 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.201 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.201 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.201 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.201 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.201 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.201 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.202 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.202 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.202 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.202 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.202 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.203 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.203 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.203 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.203 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83568/exe 2023-10-12T02:14:48.204 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.204 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_aio.log 2023-10-12T02:14:48.204 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_lock 2023-10-12T02:14:48.204 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83640 2023-10-12T02:14:48.204 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_lock on pid 83640' 2023-10-12T02:14:48.205 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2023-10-12T02:14:48.205 INFO:tasks.workunit.client.0.smithi125.stdout:test api_lock on pid 83640 2023-10-12T02:14:48.213 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83640 2023-10-12T02:14:48.213 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.213 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.213 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.214 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.214 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.214 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.214 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.214 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.215 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.215 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.215 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.215 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.215 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.216 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.216 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.216 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.216 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.216 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.217 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.217 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.217 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.217 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.217 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.218 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_lock_pp 2023-10-12T02:14:48.218 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.218 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.218 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.218 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.219 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.219 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.219 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.219 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.219 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.220 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.220 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.220 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.220 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.220 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.221 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.221 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.221 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_lock_pp' 2023-10-12T02:14:48.221 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.221 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.222 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.222 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.222 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.222 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.222 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.223 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.223 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.223 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.223 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.223 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.223 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.224 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.224 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.224 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.224 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.224 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.225 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.225 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.225 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.225 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_aio_pp 2023-10-12T02:14:48.225 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.225 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.226 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.226 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.226 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.226 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.226 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.227 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.227 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.227 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_lock_pp 2023-10-12T02:14:48.227 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.227 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.228 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.228 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.228 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.228 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.228 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.228 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.229 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.229 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.229 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.229 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.229 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.230 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.230 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.230 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.230 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.230 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.231 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_aio_pp.log 2023-10-12T02:14:48.231 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.231 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.231 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.231 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.232 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_aio_pp: /' 2023-10-12T02:14:48.232 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.232 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.232 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.232 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.233 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.233 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.233 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_lock_pp 2023-10-12T02:14:48.233 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83660 2023-10-12T02:14:48.233 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.233 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.234 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2023-10-12T02:14:48.234 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.234 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.234 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_lock_pp on pid 83660' 2023-10-12T02:14:48.234 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.235 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.235 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.235 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.236 INFO:tasks.workunit.client.0.smithi125.stdout:test api_lock_pp on pid 83660 2023-10-12T02:14:48.251 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83660 2023-10-12T02:14:48.252 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.252 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.252 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.252 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.252 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.253 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.253 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.253 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.253 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.253 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.254 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.254 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.254 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.254 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.254 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.255 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.255 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.255 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.255 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.255 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.256 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.256 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.256 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.256 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.256 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.257 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.257 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.257 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.257 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.257 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.258 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.258 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.258 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.258 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.258 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.259 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.259 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.259 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.259 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.259 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.259 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.260 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.260 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.260 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.260 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.260 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.261 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.261 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.261 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.261 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.261 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.262 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.262 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.262 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.262 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.262 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.263 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.263 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.263 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.263 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.263 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.264 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.264 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.264 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.264 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.264 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.265 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.265 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.265 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.265 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.265 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.265 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.266 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.266 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.266 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.266 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.266 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.267 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.267 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.267 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.267 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.267 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.268 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.268 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.268 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.268 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.268 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.269 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.269 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.269 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.269 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.269 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.269 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.270 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.270 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.270 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.270 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.270 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83588/exe 2023-10-12T02:14:48.271 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83577/exe 2023-10-12T02:14:48.271 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.271 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.271 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.271 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.272 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.272 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.272 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.272 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.272 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.272 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.273 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.273 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.273 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.273 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.273 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.274 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.274 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.274 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.274 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.274 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.275 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.275 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.275 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.275 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.275 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.275 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.276 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.276 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.276 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.276 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.276 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.277 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.277 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.277 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.277 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.277 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.277 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.278 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.278 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.278 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.278 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.278 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.279 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.279 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.279 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.279 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.279 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.280 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.280 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.280 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.280 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.280 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.281 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.281 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.281 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.281 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.281 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.281 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.282 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.282 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.282 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.282 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_misc 2023-10-12T02:14:48.282 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.283 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_io 2023-10-12T02:14:48.283 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.283 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.283 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.283 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_misc' 2023-10-12T02:14:48.284 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.284 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_io_pp 2023-10-12T02:14:48.284 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.284 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.284 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.285 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.285 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.285 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_io_pp.log 2023-10-12T02:14:48.285 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.285 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.286 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.286 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.286 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.286 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.286 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.287 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_io: /' 2023-10-12T02:14:48.287 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.287 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_io_pp: /' 2023-10-12T02:14:48.287 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.287 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.287 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.288 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.288 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.288 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.288 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.288 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.289 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.289 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.289 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.289 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.289 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.289 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.290 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.290 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.290 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.290 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.290 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.291 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.291 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.291 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.291 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.291 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.292 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.292 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.292 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.292 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.292 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_io.log 2023-10-12T02:14:48.293 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.293 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_misc 2023-10-12T02:14:48.293 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.293 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83603/exe 2023-10-12T02:14:48.293 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_misc 2023-10-12T02:14:48.293 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.294 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83693 2023-10-12T02:14:48.294 INFO:tasks.workunit.client.0.smithi125.stdout:test api_misc on pid 83693 2023-10-12T02:14:48.299 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_misc on pid 83693' 2023-10-12T02:14:48.300 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83693 2023-10-12T02:14:48.300 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.300 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.300 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2023-10-12T02:14:48.300 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_misc_pp 2023-10-12T02:14:48.301 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_misc_pp' 2023-10-12T02:14:48.301 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.301 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.301 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.301 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.302 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_misc_pp 2023-10-12T02:14:48.302 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.302 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.302 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.302 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.303 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.303 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.303 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.303 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.303 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.304 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.304 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_asio.log 2023-10-12T02:14:48.304 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.304 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.304 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_asio: /' 2023-10-12T02:14:48.305 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.305 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.305 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.305 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.305 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.305 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.306 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.306 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.306 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.306 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.306 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.307 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.307 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.307 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.307 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.307 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.308 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.308 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.308 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_asio 2023-10-12T02:14:48.308 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.308 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.309 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.309 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.309 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.309 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.309 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.309 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.310 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.310 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.310 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_misc_pp 2023-10-12T02:14:48.310 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.310 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.311 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.311 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.311 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.311 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83706 2023-10-12T02:14:48.311 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_misc_pp on pid 83706' 2023-10-12T02:14:48.312 INFO:tasks.workunit.client.0.smithi125.stdout:test api_misc_pp on pid 83706 2023-10-12T02:14:48.331 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83706 2023-10-12T02:14:48.331 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.332 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.332 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2023-10-12T02:14:48.332 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.332 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.332 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.333 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.333 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.333 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.333 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_tier_pp 2023-10-12T02:14:48.333 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_tier_pp' 2023-10-12T02:14:48.334 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.334 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.334 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.334 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.334 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.335 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.335 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.335 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.335 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.335 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.336 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.336 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.336 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.336 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.336 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.337 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.337 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.337 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.337 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.337 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.337 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.338 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.338 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.338 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.338 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.338 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.339 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.339 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.339 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.339 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.339 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.340 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.340 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.340 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.340 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.340 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.341 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.341 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.341 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.341 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.341 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.341 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.342 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.342 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.342 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.342 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.342 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.343 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.343 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.343 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.343 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.343 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.344 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.344 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.344 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.344 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.344 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.345 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.345 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.345 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.345 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.345 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.345 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.346 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.346 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83640/exe 2023-10-12T02:14:48.346 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.346 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.346 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.347 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.347 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.347 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.347 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.347 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.348 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.348 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.348 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.348 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.348 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.349 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.349 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.349 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.349 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.349 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.349 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.350 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.350 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.350 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.350 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.350 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.351 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.351 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.351 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.351 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.351 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.352 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.352 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.352 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.352 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.352 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.353 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.353 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.353 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.353 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.353 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.353 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.354 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.354 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.354 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.354 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.354 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.355 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.355 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.355 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.355 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.355 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.356 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.356 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.356 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_lock.log 2023-10-12T02:14:48.356 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.356 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.357 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.357 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.357 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.357 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.357 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.357 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.358 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.358 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.358 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.358 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.358 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.359 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.359 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.359 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.359 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.359 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.360 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.360 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.360 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_lock 2023-10-12T02:14:48.360 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.360 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.360 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.361 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.361 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.361 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.361 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.361 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.362 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.362 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.362 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.362 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.362 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.363 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.363 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.363 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.363 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.363 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.364 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.364 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.364 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.364 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.364 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.365 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.365 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83620/exe 2023-10-12T02:14:48.365 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.365 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.365 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.365 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.366 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.366 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.366 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.366 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.366 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.367 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.367 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.367 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.367 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.367 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.368 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.368 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.368 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.368 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.368 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.368 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.369 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.369 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.369 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.369 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.369 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.370 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.370 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.370 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.370 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.370 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.371 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83660/exe 2023-10-12T02:14:48.371 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.371 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.371 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.371 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.371 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_list: /' 2023-10-12T02:14:48.372 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.372 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_tier_pp 2023-10-12T02:14:48.372 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.372 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.372 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.373 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.373 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.373 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.373 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.373 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.374 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.374 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.374 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_lock_pp.log 2023-10-12T02:14:48.374 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_lock_pp 2023-10-12T02:14:48.374 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_lock: /' 2023-10-12T02:14:48.374 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_list 2023-10-12T02:14:48.375 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.375 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.375 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.375 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.375 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.376 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.376 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.376 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.376 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.376 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.377 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.377 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.377 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.377 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.377 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.377 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.378 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.378 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.378 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_lock_pp: /' 2023-10-12T02:14:48.378 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.378 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_list.log 2023-10-12T02:14:48.379 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_tier_pp 2023-10-12T02:14:48.379 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83753 2023-10-12T02:14:48.379 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_tier_pp on pid 83753' 2023-10-12T02:14:48.379 INFO:tasks.workunit.client.0.smithi125.stdout:test api_tier_pp on pid 83753 2023-10-12T02:14:48.390 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83753 2023-10-12T02:14:48.390 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.390 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.391 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2023-10-12T02:14:48.391 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.391 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.391 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.391 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.392 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.392 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.392 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.392 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.392 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.393 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.393 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.393 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.393 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.393 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.394 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.394 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.394 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.394 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.394 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.395 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.395 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.395 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.395 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.395 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.396 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.396 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.396 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.396 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.396 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.397 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.397 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.397 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.397 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.397 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.398 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.398 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.398 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.398 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.398 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.399 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.399 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.399 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.399 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.399 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.399 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.400 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.400 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.400 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.400 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.400 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.401 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.401 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.401 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.401 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.401 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.402 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.402 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.402 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.402 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.402 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_pool 2023-10-12T02:14:48.402 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_pool' 2023-10-12T02:14:48.403 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.403 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.403 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.403 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.403 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.404 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.404 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.404 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.404 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.404 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.405 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.405 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.405 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.405 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.405 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.405 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.406 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.406 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.406 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.406 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.406 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.407 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.407 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.407 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.407 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_pool 2023-10-12T02:14:48.407 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.408 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.408 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.408 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.408 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.408 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.409 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.409 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.409 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.409 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.409 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.409 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.410 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.410 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.410 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.410 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.410 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.411 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.411 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.411 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.411 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.411 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.412 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.412 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.412 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.412 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.412 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.412 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.413 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.413 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.413 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.413 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.413 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.414 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.414 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.414 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.414 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.414 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.415 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.415 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.415 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.415 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.415 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.415 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.416 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.416 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.416 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.416 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.416 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.417 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.417 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.417 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.417 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.417 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.418 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.418 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.418 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.418 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.418 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.418 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.419 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.419 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.419 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83693/exe 2023-10-12T02:14:48.419 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_pool 2023-10-12T02:14:48.419 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83817 2023-10-12T02:14:48.420 INFO:tasks.workunit.client.0.smithi125.stdout:test api_pool on pid 83817 2023-10-12T02:14:48.422 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_pool on pid 83817' 2023-10-12T02:14:48.422 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83817 2023-10-12T02:14:48.422 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.423 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.423 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.423 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_snapshots 2023-10-12T02:14:48.423 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_snapshots' 2023-10-12T02:14:48.423 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2023-10-12T02:14:48.424 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.424 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.424 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.424 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.424 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.424 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.425 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.425 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.425 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.425 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.426 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.151+0000 7f40cf853640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:48.426 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2023-10-12T02:14:48.426 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [----------] Global test environment set-up. 2023-10-12T02:14:48.426 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [----------] 12 tests from AsioRados 2023-10-12T02:14:48.427 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.164+0000 7f40cf853640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:48.427 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.164+0000 7f40cf853640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:48.427 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.164+0000 7f40cf853640 1 Processor -- start 2023-10-12T02:14:48.427 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.165+0000 7f40cf853640 1 -- start start 2023-10-12T02:14:48.427 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.166+0000 7f40cf853640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380ef07cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:14:48.428 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.166+0000 7f40cf853640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380ee02010 con 0x56380ee737b0 2023-10-12T02:14:48.428 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.166+0000 7f40c899a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380ef07cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:14:48.428 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.168+0000 7f40c899a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380ef07cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42268/0 (socket says 172.21.15.125:42268) 2023-10-12T02:14:48.428 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.168+0000 7f40c899a640 1 -- 172.21.15.125:0/757791685 learned_addr learned my addr 172.21.15.125:0/757791685 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:14:48.428 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.168+0000 7f40c899a640 1 -- 172.21.15.125:0/757791685 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380eda9630 con 0x56380ee737b0 2023-10-12T02:14:48.429 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.168+0000 7f40c899a640 1 --2- 172.21.15.125:0/757791685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380ef07cb0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a18663dc760cbd server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:14:48.429 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.168+0000 7f40c7197640 1 -- 172.21.15.125:0/757791685 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40b8006a20 con 0x56380ee737b0 2023-10-12T02:14:48.429 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.168+0000 7f40c7197640 1 -- 172.21.15.125:0/757791685 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f40b80062a0 con 0x56380ee737b0 2023-10-12T02:14:48.429 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.168+0000 7f40c7197640 1 -- 172.21.15.125:0/757791685 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40b80014e0 con 0x56380ee737b0 2023-10-12T02:14:48.429 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.172+0000 7f40cf853640 1 -- 172.21.15.125:0/757791685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 msgr2=0x56380ef07cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:14:48.430 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.172+0000 7f40cf853640 1 --2- 172.21.15.125:0/757791685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380ef07cb0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:14:48.430 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.172+0000 7f40cf853640 1 -- 172.21.15.125:0/757791685 shutdown_connections 2023-10-12T02:14:48.432 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.432 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.433 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.433 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.433 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.433 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.433 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.434 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.434 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.434 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.434 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.434 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.435 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.435 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.435 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.435 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.435 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.436 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_misc: /' 2023-10-12T02:14:48.436 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.436 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_misc.log 2023-10-12T02:14:48.436 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_snapshots 2023-10-12T02:14:48.436 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_misc 2023-10-12T02:14:48.437 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83706/exe 2023-10-12T02:14:48.437 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.437 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_snapshots 2023-10-12T02:14:48.437 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83844 2023-10-12T02:14:48.437 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_snapshots on pid 83844' 2023-10-12T02:14:48.438 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.172+0000 7f40ctest api_snapshots on pid 83844 2023-10-12T02:14:48.442 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83844 2023-10-12T02:14:48.443 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.443 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.443 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.443 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.444 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.444 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.444 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2023-10-12T02:14:48.444 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.444 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.445 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.445 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.445 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.445 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.445 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.446 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.446 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.446 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.446 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.446 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.446 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.447 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.447 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.447 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.447 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.447 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.448 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_misc_pp: /' 2023-10-12T02:14:48.448 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_misc_pp 2023-10-12T02:14:48.448 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.448 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_misc_pp.log 2023-10-12T02:14:48.448 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_snapshots_pp 2023-10-12T02:14:48.449 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_snapshots_pp' 2023-10-12T02:14:48.449 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.449 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.449 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.449 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.450 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.450 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.450 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.450 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.450 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.451 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.451 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.451 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.451 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.451 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.452 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.452 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.452 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.452 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.452 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.452 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.453 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_snapshots_pp 2023-10-12T02:14:48.453 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_snapshots_pp 2023-10-12T02:14:48.453 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83869 2023-10-12T02:14:48.453 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_snapshots_pp on pid 83869' 2023-10-12T02:14:48.454 INFO:tasks.workunit.client.0.smithi125.stdout:test api_snapshots_pp on pid 83869 2023-10-12T02:14:48.462 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83869 2023-10-12T02:14:48.462 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.463 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.463 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_stat 2023-10-12T02:14:48.463 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_stat' 2023-10-12T02:14:48.463 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.463 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.464 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.464 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.464 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.464 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.464 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.465 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.465 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.465 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.465 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.465 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.466 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.466 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.466 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.466 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.466 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.467 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.467 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.467 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.467 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2023-10-12T02:14:48.467 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.468 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.468 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.468 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.468 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.468 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.468 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.469 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.469 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.469 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.469 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.469 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.470 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.470 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.470 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.470 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.470 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.471 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.471 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.471 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_stat 2023-10-12T02:14:48.471 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.471 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.471 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.472 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.472 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.472 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.472 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.472 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.473 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.473 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.473 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.473 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.473 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.474 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.474 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.474 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.474 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.474 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.474 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.475 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.475 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.475 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.475 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.475 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.476 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.476 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.476 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.476 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.476 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.477 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.477 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.477 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.477 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.477 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.477 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83753/exe 2023-10-12T02:14:48.478 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.478 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.478 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.478 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.478 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.479 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.479 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.479 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.479 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.479 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.480 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.480 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.480 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.480 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.480 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.481 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.481 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.481 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.481 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.481 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.481 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.482 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.482 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.482 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_stat 2023-10-12T02:14:48.482 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.482 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.483 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.483 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.483 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.483 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.483 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83895 2023-10-12T02:14:48.484 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_stat on pid 83895' 2023-10-12T02:14:48.484 INFO:tasks.workunit.client.0.smithi125.stdout:test api_stat on pid 83895 2023-10-12T02:14:48.488 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83895 2023-10-12T02:14:48.488 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.488 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2023-10-12T02:14:48.488 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.488 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.489 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.489 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.489 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.489 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.489 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.490 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.490 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.490 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.490 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.490 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_stat_pp 2023-10-12T02:14:48.491 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_stat_pp' 2023-10-12T02:14:48.491 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.491 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.491 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.491 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.492 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.492 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.492 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.492 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.492 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.492 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.493 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.493 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.493 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.493 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.493 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.494 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.494 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.494 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.494 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.494 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.495 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.495 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.495 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.495 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.495 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.496 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.496 INFO:tasks.workunit.client.0.smithi125.stdout:f853640 1 --2- 172.21.15.125:0/757791685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380ef07cb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:14:48.496 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.172+0000 7f40cf853640 1 -- 172.21.15.125:0/757791685 >> 172.21.15.125:0/757791685 conn(0x56380ed7cd80 msgr2=0x56380ef91ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:14:48.496 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.172+0000 7f40cf853640 1 -- 172.21.15.125:0/757791685 shutdown_connections 2023-10-12T02:14:48.497 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.172+0000 7f40cf853640 1 -- 172.21.15.125:0/757791685 wait complete. 2023-10-12T02:14:48.497 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.173+0000 7f40cf853640 1 Processor -- start 2023-10-12T02:14:48.497 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.173+0000 7f40cf853640 1 -- start start 2023-10-12T02:14:48.497 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.174+0000 7f40cf853640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380eef1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:14:48.497 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.174+0000 7f40cf853640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380ed9d1e0 con 0x56380ee737b0 2023-10-12T02:14:48.498 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.175+0000 7f40c899a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380eef1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:14:48.498 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.175+0000 7f40c899a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380eef1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42284/0 (socket says 172.21.15.125:42284) 2023-10-12T02:14:48.498 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.175+0000 7f40c899a640 1 -- 172.21.15.125:0/1696364954 learned_addr learned my addr 172.21.15.125:0/1696364954 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-12T02:14:48.498 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.176+0000 7f40c899a640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380ed9c7e0 con 0x56380ee737b0 2023-10-12T02:14:48.498 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.176+0000 7f40c899a640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380eef1930 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:14:48.499 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.197+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40b802aa00 con 0x56380ee737b0 2023-10-12T02:14:48.499 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.197+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f40b802ab60 con 0x56380ee737b0 2023-10-12T02:14:48.499 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.197+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40b802ae00 con 0x56380ee737b0 2023-10-12T02:14:48.499 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.197+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56380ed9c240 con 0x56380ee737b0 2023-10-12T02:14:48.499 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.198+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56380eef4dc0 con 0x56380ee737b0 2023-10-12T02:14:48.499 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.201+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 44855+0+0 (crc 0 0 0) 0x7f40b8038410 con 0x56380ee737b0 2023-10-12T02:14:48.514 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_tier_pp 2023-10-12T02:14:48.514 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.515 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.515 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.515 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.515 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.515 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.516 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.516 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.516 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.516 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.516 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.517 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.517 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.517 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.517 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.517 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.518 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.518 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.518 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.518 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.518 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.519 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.519 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.519 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.519 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.519 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.520 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.520 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.520 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.520 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_tier_pp.log 2023-10-12T02:14:48.520 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_tier_pp: /' 2023-10-12T02:14:48.521 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.521 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.521 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.521 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.521 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.522 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.522 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.522 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.522 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.522 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.523 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.523 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.523 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.523 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.523 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.523 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.524 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.524 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.524 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.524 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.524 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.525 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.525 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.525 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.525 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.525 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.526 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.526 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.526 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.526 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.526 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.527 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.527 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.527 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.527 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_stat_pp 2023-10-12T02:14:48.527 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.528 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.528 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.528 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.528 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.528 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.529 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.529 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.529 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.529 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.529 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.530 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.530 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.530 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.530 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.530 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.531 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.531 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.531 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.531 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.531 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.532 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.532 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.532 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.532 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.532 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.532 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.533 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.533 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.533 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.533 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.533 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.533 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.534 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.534 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.534 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.534 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.534 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.535 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.535 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.535 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.535 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.535 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.536 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.536 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.536 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.536 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.536 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.537 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.537 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.537 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.537 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.537 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.538 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.538 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.538 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.538 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.538 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.539 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.539 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.539 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.539 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.539 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.540 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.540 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.540 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.540 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.540 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.540 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.541 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.541 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.541 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.541 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.541 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83869/exe 2023-10-12T02:14:48.542 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.542 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.542 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.542 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.542 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.543 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.543 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.543 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.543 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.543 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.543 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.544 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.544 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.544 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.544 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.544 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.545 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.545 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.545 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.545 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.545 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.546 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.546 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.546 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.546 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.546 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.546 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.547 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.547 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.547 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.547 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.547 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.548 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.548 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.548 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.548 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.548 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.549 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.549 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.549 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.549 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.549 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.549 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.550 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_stat_pp 2023-10-12T02:14:48.550 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.550 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83844/exe 2023-10-12T02:14:48.550 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83955 2023-10-12T02:14:48.550 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.551 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.551 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.551 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.551 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.551 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.552 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.552 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_stat_pp on pid 83955' 2023-10-12T02:14:48.552 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:test api_stat_pp on pid 83955 2023-10-12T02:14:48.558 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83955 2023-10-12T02:14:48.559 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.559 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.559 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.559 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.560 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.560 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.560 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.560 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_watch_notify 2023-10-12T02:14:48.560 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.561 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.561 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.561 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_snapshots_pp: /' 2023-10-12T02:14:48.561 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.561 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_watch_notify' 2023-10-12T02:14:48.562 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.562 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.562 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.562 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.562 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.563 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2023-10-12T02:14:48.563 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.563 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.563 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.563 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.564 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2023-10-12T02:14:48.564 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.564 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.564 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.564 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.565 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.565 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.565 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.565 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_snapshots_pp 2023-10-12T02:14:48.565 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.565 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.566 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.566 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.566 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.566 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.566 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.567 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.567 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.567 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_snapshots 2023-10-12T02:14:48.567 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_snapshots: /' 2023-10-12T02:14:48.567 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83817/exe 2023-10-12T02:14:48.568 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_watch_notify 2023-10-12T02:14:48.568 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_snapshots.log 2023-10-12T02:14:48.568 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.568 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.568 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.569 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.569 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.569 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.569 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.569 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.570 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.570 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.570 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.570 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.571 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.571 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.571 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.571 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.571 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.572 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.572 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.572 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.572 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.572 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.573 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.573 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.573 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.573 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.573 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.574 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.574 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.574 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.574 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.574 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.575 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.575 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.575 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.575 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.575 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.576 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.576 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_watch_notify 2023-10-12T02:14:48.576 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.576 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.576 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.577 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.577 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.577 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.577 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83980 2023-10-12T02:14:48.577 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_watch_notify on pid 83980' 2023-10-12T02:14:48.578 INFO:tasks.workunit.client.0.smithi125.stdout:test api_watch_notify on pid 83980 2023-10-12T02:14:48.589 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=83980 2023-10-12T02:14:48.590 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.590 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.590 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.590 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.590 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.591 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.591 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.591 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.591 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.591 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_pool.log 2023-10-12T02:14:48.592 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_pool: /' 2023-10-12T02:14:48.592 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.592 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.592 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.592 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.593 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.593 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.593 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.593 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.593 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.594 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.594 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.594 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.594 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.594 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.594 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.595 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.595 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.595 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.595 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.595 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.596 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2023-10-12T02:14:48.596 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_watch_notify_pp 2023-10-12T02:14:48.596 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.596 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.596 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_watch_notify_pp' 2023-10-12T02:14:48.597 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.597 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.597 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.597 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.597 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_pool 2023-10-12T02:14:48.598 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.598 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.598 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.598 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.598 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.599 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.599 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.599 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.599 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.599 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.599 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.600 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.600 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.600 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.600 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.600 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.601 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.601 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.601 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.601 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.601 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.602 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.602 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.602 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.602 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.602 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.603 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.603 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.603 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.603 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.603 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.603 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.604 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.604 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.604 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.604 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.604 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.605 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.605 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.605 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.605 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.605 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.606 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.606 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.606 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.606 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.606 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.606 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.607 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.607 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.607 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.607 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.607 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.608 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.608 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.608 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.608 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.608 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.609 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.609 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.609 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.609 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.609 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.609 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.610 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.610 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.610 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.610 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.610 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.611 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.611 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.611 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.611 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.611 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.612 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.612 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.612 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.612 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.612 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.612 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.613 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.613 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.613 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83955/exe 2023-10-12T02:14:48.613 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.613 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.614 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.614 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.614 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.614 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.614 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.615 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.615 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.615 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.615 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.615 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.616 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.616 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.616 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_watch_notify_pp 2023-10-12T02:14:48.616 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.616 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.617 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.617 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.617 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.617 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.617 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.617 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.618 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.618 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.618 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.618 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.618 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_watch_notify_pp 2023-10-12T02:14:48.619 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84033 2023-10-12T02:14:48.619 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_watch_notify_pp on pid 84033' 2023-10-12T02:14:48.619 INFO:tasks.workunit.client.0.smithi125.stdout:test api_watch_notify_pp on pid 84033 2023-10-12T02:14:48.626 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84033 2023-10-12T02:14:48.626 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.626 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.626 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.626 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.627 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.627 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.627 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.627 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.627 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2023-10-12T02:14:48.628 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_cmd 2023-10-12T02:14:48.628 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_cmd' 2023-10-12T02:14:48.628 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.628 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.628 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.629 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.629 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.629 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.629 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_stat_pp 2023-10-12T02:14:48.629 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.630 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.630 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.630 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.630 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.630 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.630 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.631 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.631 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.631 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.631 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.631 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.632 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.632 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.632 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.632 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.632 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.633 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.633 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.633 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.633 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.633 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.633 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.634 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.634 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.634 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.634 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.634 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.635 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.635 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.635 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.635 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.635 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.636 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.636 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.636 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.636 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.636 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.637 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.637 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.637 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.637 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.637 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.637 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.638 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.638 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_stat_pp.log 2023-10-12T02:14:48.638 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_stat_pp: /' 2023-10-12T02:14:48.638 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_cmd 2023-10-12T02:14:48.638 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.639 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.639 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.639 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.639 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.639 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.640 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.640 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.640 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.640 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_cmd 2023-10-12T02:14:48.640 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.640 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.641 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.641 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.641 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84058 2023-10-12T02:14:48.641 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_cmd on pid 84058' 2023-10-12T02:14:48.642 INFO:tasks.workunit.client.0.smithi125.stdout:test api_cmd on pid 84058 2023-10-12T02:14:48.655 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84058 2023-10-12T02:14:48.655 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.655 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.656 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.656 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.656 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.656 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.656 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.657 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_cmd_pp 2023-10-12T02:14:48.657 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.657 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.657 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2023-10-12T02:14:48.657 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.657 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.658 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.658 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.658 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.658 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.659 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.659 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.659 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.659 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.659 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.660 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.660 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.660 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.660 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.660 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.660 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.661 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.661 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_cmd_pp' 2023-10-12T02:14:48.661 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.661 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.661 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.662 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.662 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.662 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.662 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.662 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.663 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.663 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.663 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.663 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.663 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.664 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.664 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.664 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.664 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.664 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.665 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.665 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.665 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.665 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.665 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.666 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.666 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.666 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.666 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.666 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.666 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.667 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.667 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.667 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.667 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.667 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.668 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.668 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.668 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.668 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.668 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.669 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.669 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.669 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.669 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.669 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.670 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.670 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.670 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.670 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.670 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.670 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.671 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.671 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.671 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.671 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.671 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.672 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.672 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.672 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.672 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.672 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.673 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.673 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.673 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.673 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.673 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.674 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.674 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.674 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.674 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.674 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.674 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.675 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.675 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.675 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.675 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.675 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.676 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.676 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.676 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.676 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.676 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.677 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.677 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.677 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.677 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.677 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.678 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_cmd_pp 2023-10-12T02:14:48.678 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.678 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.678 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83980/exe 2023-10-12T02:14:48.678 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.678 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.679 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.679 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.679 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.679 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.679 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.680 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.680 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.680 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.680 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.680 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.681 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.681 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.681 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.681 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.681 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.681 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.682 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.682 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.682 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.682 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.682 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.683 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.683 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.683 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.683 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.683 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.684 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.684 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.684 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.684 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.684 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.685 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.685 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.685 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.685 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.685 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.686 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.686 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.686 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.686 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/83895/exe 2023-10-12T02:14:48.686 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_cmd_pp 2023-10-12T02:14:48.687 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84094 2023-10-12T02:14:48.687 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.687 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.687 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2023-10-12T02:14:48.687 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_cmd_pp on pid 84094' 2023-10-12T02:14:48.688 INFO:tasks.workunit.client.0.smithi125.stdout:test api_cmd_pp on pid 84094 2023-10-12T02:14:48.695 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.695 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84094 2023-10-12T02:14:48.695 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.695 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.695 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.696 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.696 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.696 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.696 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.696 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.697 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.697 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_watch_notify 2023-10-12T02:14:48.697 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.697 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.697 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.698 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_service 2023-10-12T02:14:48.698 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.698 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.698 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.698 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.699 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_service' 2023-10-12T02:14:48.699 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_watch_notify: /' 2023-10-12T02:14:48.699 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.699 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.699 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.699 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.700 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_watch_notify.log 2023-10-12T02:14:48.700 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.700 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.700 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.700 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.701 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.701 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.701 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_stat 2023-10-12T02:14:48.701 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_stat: /' 2023-10-12T02:14:48.701 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.702 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.702 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.702 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.702 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.702 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_stat.log 2023-10-12T02:14:48.703 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.703 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.703 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.703 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.703 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.703 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.704 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.704 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.704 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.704 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.704 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.705 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.705 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.705 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.705 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.705 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.706 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.706 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.706 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.706 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.706 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.706 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.707 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.707 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.707 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.707 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_service 2023-10-12T02:14:48.707 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.708 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.708 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.708 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.708 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.708 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.709 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.709 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.709 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.709 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.709 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.710 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.710 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.710 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.710 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.710 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.711 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_service 2023-10-12T02:14:48.711 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84115 2023-10-12T02:14:48.711 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_service on pid 84115' 2023-10-12T02:14:48.711 INFO:tasks.workunit.client.0.smithi125.stdout:test api_service on pid 84115 2023-10-12T02:14:48.729 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84115 2023-10-12T02:14:48.729 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.729 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.730 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.730 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.730 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.730 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.730 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_service_pp 2023-10-12T02:14:48.731 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.731 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.731 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.731 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.731 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.732 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.732 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.732 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.732 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.732 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.732 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.733 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.733 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.733 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.733 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.733 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.734 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.734 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_service_pp' 2023-10-12T02:14:48.734 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_service_pp 2023-10-12T02:14:48.734 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.734 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.735 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.735 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.735 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.735 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.735 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.736 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.736 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.736 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.736 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.736 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.737 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.737 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.737 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.737 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.737 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.737 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.738 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.738 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.738 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.738 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.738 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.739 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.739 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.739 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.739 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.739 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.740 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.740 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.740 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.740 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.740 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.741 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.741 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.741 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.741 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.741 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.741 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.742 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.742 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.742 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.742 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.742 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.743 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.743 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.743 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.743 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.743 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.744 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.744 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.744 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.744 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.744 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.745 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.745 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84058/exe 2023-10-12T02:14:48.745 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84033/exe 2023-10-12T02:14:48.745 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.745 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2023-10-12T02:14:48.746 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.746 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.746 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.746 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.746 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.747 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.747 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.747 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.747 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.747 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.748 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.748 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.748 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.748 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.748 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.748 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.749 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.749 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.749 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.749 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.749 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.750 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.750 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.750 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.750 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_cmd 2023-10-12T02:14:48.750 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_cmd.log 2023-10-12T02:14:48.751 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.751 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.751 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.751 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.751 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.752 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.752 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.752 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_watch_notify_pp 2023-10-12T02:14:48.752 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.752 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.752 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.753 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_cmd: /' 2023-10-12T02:14:48.753 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.753 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.753 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.753 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.754 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.754 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.754 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.754 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.754 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.755 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.755 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.755 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.755 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.755 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.756 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.756 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.756 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.756 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.756 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.756 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.757 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.757 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.757 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.757 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.757 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.758 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.758 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.758 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.758 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.758 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.759 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.759 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.759 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.759 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.759 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.760 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.760 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.760 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.760 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.760 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.761 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.761 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.761 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.761 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.761 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.762 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.762 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.762 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.762 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.762 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.763 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.763 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.763 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.763 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.763 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.764 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.764 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.764 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.764 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.764 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.764 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.765 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.765 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.765 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.765 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.765 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_service_pp 2023-10-12T02:14:48.766 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84159 2023-10-12T02:14:48.766 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_service_pp on pid 84159' 2023-10-12T02:14:48.766 INFO:tasks.workunit.client.0.smithi125.stdout:test api_service_pp on pid 84159 2023-10-12T02:14:48.775 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84159 2023-10-12T02:14:48.775 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.776 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.776 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2023-10-12T02:14:48.776 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.776 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.776 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.777 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.777 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.777 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.777 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.777 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.778 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.778 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.778 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.778 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.778 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.779 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.779 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.779 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.779 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.779 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.780 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.780 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.780 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.780 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.780 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.780 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.781 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.781 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.781 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.781 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.782 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.782 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.782 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.782 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.782 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.782 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.783 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2023-10-12T02:14:48.783 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2023-10-12T02:14:48.783 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.783 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.783 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.784 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.784 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.784 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.784 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.784 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.785 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.785 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.785 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.785 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.785 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.786 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.786 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.786 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.786 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.787 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.787 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.787 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.787 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.787 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_c_write_operations 2023-10-12T02:14:48.788 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_c_write_operations' 2023-10-12T02:14:48.788 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_c_write_operations 2023-10-12T02:14:48.788 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.788 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.788 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.789 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.789 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.789 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.789 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.789 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.790 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.790 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.790 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.790 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.790 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.791 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.791 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.791 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.791 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.791 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.792 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.792 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.792 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.792 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.792 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84094/exe 2023-10-12T02:14:48.793 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.793 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.793 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.793 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.793 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.793 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.794 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.794 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.794 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.794 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.794 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.795 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.795 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.795 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.795 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.795 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.796 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.796 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.796 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_c_write_operations 2023-10-12T02:14:48.796 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84184 2023-10-12T02:14:48.797 INFO:tasks.workunit.client.0.smithi125.stdout:test api_c_write_operations on pid 84184 2023-10-12T02:14:48.800 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_c_write_operations on pid 84184' 2023-10-12T02:14:48.801 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84184 2023-10-12T02:14:48.801 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.801 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.801 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.802 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.802 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.802 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.802 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.802 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_c_read_operations 2023-10-12T02:14:48.803 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.803 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.803 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.803 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.803 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2023-10-12T02:14:48.803 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.804 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.804 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_c_read_operations' 2023-10-12T02:14:48.804 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.804 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.805 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_c_read_operations 2023-10-12T02:14:48.805 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.805 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.805 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.805 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.805 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.806 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_cmd_pp: /' 2023-10-12T02:14:48.806 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.806 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.806 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.806 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.807 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.807 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.807 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.807 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.807 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.808 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.808 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.808 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.808 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.808 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.809 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2023-10-12T02:14:48.809 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.809 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.809 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.809 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_cmd_pp 2023-10-12T02:14:48.810 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_c_read_operations 2023-10-12T02:14:48.810 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.810 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.810 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.810 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.811 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84198 2023-10-12T02:14:48.811 INFO:tasks.workunit.client.0.smithi125.stdout:test api_c_read_operations on pid 84198 2023-10-12T02:14:48.816 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_c_read_operations on pid 84198' 2023-10-12T02:14:48.817 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.817 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84198 2023-10-12T02:14:48.817 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.817 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.817 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.818 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.818 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.818 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.818 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.818 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.819 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.819 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.819 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.819 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.819 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.820 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2023-10-12T02:14:48.820 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.820 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.820 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.820 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.821 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.821 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.821 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.821 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.821 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.822 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.822 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.822 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.822 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.822 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.823 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.823 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.823 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.823 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.823 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.823 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.824 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.824 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.824 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.824 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s api_cls_remote_reads 2023-10-12T02:14:48.824 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' api_cls_remote_reads' 2023-10-12T02:14:48.825 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.825 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo api_cls_remote_reads 2023-10-12T02:14:48.825 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.825 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.825 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.826 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.826 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.826 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.826 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.826 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.827 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.827 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.827 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.827 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.827 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.827 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.828 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.828 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.828 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.828 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.828 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.829 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.829 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.829 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.829 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=api_cls_remote_reads 2023-10-12T02:14:48.829 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84263 2023-10-12T02:14:48.830 INFO:tasks.workunit.client.0.smithi125.stdout:test api_cls_remote_reads on pid 84263 2023-10-12T02:14:48.842 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test api_cls_remote_reads on pid 84263' 2023-10-12T02:14:48.843 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84263 2023-10-12T02:14:48.843 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.843 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.843 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.843 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.844 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s list_parallel 2023-10-12T02:14:48.844 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.844 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.844 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.844 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' list_parallel' 2023-10-12T02:14:48.844 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.845 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo list_parallel 2023-10-12T02:14:48.845 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2023-10-12T02:14:48.845 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.845 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.845 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.846 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.846 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.846 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.846 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.846 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.847 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.847 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.847 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.847 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.847 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.848 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.848 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.848 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.848 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.848 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.849 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.849 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.849 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.849 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.849 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.849 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.850 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.850 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.850 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.850 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.851 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.851 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.851 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.851 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.851 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.852 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.852 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.852 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.852 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.852 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.852 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.853 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.853 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.853 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.853 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.853 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.854 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.854 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.854 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.854 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.854 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.855 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.855 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.855 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84159/exe 2023-10-12T02:14:48.855 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.856 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.856 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.856 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.857 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.857 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.857 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.857 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.858 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.858 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.858 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.859 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.859 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.859 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.859 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.860 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.860 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.860 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.860 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_service_pp 2023-10-12T02:14:48.861 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.861 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.861 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.862 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.862 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.862 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.863 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.863 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.863 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.863 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.864 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.864 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.864 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.865 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.865 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.865 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.866 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.866 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.866 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.867 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.867 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.867 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.868 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.868 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.868 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.869 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.869 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.869 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.870 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.870 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.870 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.871 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.871 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.871 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.872 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.872 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.872 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.873 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.873 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.873 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.874 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.874 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.875 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.875 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.875 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.876 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.876 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.876 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.877 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.877 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.878 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.878 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.878 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.879 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.879 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.879 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_service_pp.log 2023-10-12T02:14:48.880 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_service_pp: /' 2023-10-12T02:14:48.880 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.880 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.881 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.881 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.882 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.882 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.882 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.883 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.883 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.884 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.884 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.884 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.885 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.885 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.885 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.886 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.886 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.886 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.887 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.887 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.887 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84115/exe 2023-10-12T02:14:48.888 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=list_parallel 2023-10-12T02:14:48.888 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84300 2023-10-12T02:14:48.889 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test list_parallel on pid 84300' 2023-10-12T02:14:48.889 INFO:tasks.workunit.client.0.smithi125.stdout:test list_parallel on pid 84300 2023-10-12T02:14:48.905 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84300 2023-10-12T02:14:48.906 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.906 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.907 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.907 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.907 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2023-10-12T02:14:48.908 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.908 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.908 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.909 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.909 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.909 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.910 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.910 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.911 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.911 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.911 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.912 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.912 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.912 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.913 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.913 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.913 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_service 2023-10-12T02:14:48.914 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.914 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.915 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_service.log 2023-10-12T02:14:48.915 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.915 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.916 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.916 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.916 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.917 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.917 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.918 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.918 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.918 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s open_pools_parallel 2023-10-12T02:14:48.919 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_service: /' 2023-10-12T02:14:48.919 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.919 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.920 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.920 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.921 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.921 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' open_pools_parallel' 2023-10-12T02:14:48.921 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.922 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.922 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.922 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.923 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.923 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.924 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:48.925 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.925 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.925 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.926 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.926 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:48.926 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:48.926 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.926 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.927 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo open_pools_parallel 2023-10-12T02:14:48.927 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.927 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.927 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:48.927 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.928 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:48.928 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:48.928 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:48.928 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:48.928 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:48.929 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.929 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:48.929 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:48.929 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:48.929 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:48.930 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:48.930 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.930 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:48.930 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.930 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:48.931 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:48.931 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.931 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:48.931 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.931 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:48.932 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=open_pools_parallel 2023-10-12T02:14:48.932 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:48.932 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84331 2023-10-12T02:14:48.932 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test open_pools_parallel on pid 84331' 2023-10-12T02:14:48.933 INFO:tasks.workunit.client.0.smithi125.stdout:test open_pools_parallel on pid 84331 2023-10-12T02:14:48.943 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84331 2023-10-12T02:14:48.943 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2023-10-12T02:14:48.943 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:48.944 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.944 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.944 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.944 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.944 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.945 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.945 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.945 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.945 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.945 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.946 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.946 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84184/exe 2023-10-12T02:14:48.946 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.946 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.946 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.947 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.947 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.947 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.947 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.947 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.947 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.948 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2023-10-12T02:14:48.948 INFO:tasks.workunit.client.0.smithi125.stderr:++ printf %25s delete_pools_parallel 2023-10-12T02:14:48.948 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:48.948 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:48.948 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:48.949 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.949 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:48.949 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.949 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:48.949 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:48.950 INFO:tasks.workunit.client.0.smithi125.stderr:+ r=' delete_pools_parallel' 2023-10-12T02:14:48.950 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:48.950 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.950 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:48.950 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:48.951 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.951 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:48.951 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:48.951 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:48.951 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.952 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:48.952 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.952 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:48.952 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.952 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:48.952 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:48.953 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:48.953 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:48.953 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:48.953 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:48.953 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:48.954 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.954 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:48.954 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.954 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:48.954 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:48.955 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:48.955 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:48.955 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.955 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:48.955 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.956 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:48.956 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:48.956 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:48.956 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:48.956 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.957 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:48.957 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.957 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:48.957 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:48.957 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:48.957 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.958 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:48.958 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:48.958 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:48.958 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:48.958 INFO:tasks.workunit.client.0.smithi125.stderr:++ echo delete_pools_parallel 2023-10-12T02:14:48.959 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:48.959 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:48.959 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.959 INFO:tasks.workunit.client.0.smithi125.stderr:++ awk '{print $1}' 2023-10-12T02:14:48.959 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.960 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.960 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.960 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.960 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.960 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.961 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.961 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.961 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_c_write_operations: /' 2023-10-12T02:14:48.961 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2023-10-12T02:14:48.961 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:48.962 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:48.962 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:48.962 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:48.962 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.962 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:48.963 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.963 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:48.963 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:48.963 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.963 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:48.964 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:48.964 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.964 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:48.964 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.964 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:48.964 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:48.965 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.965 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:48.965 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.965 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:48.965 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.966 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:48.966 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:48.966 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:48.966 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:48.966 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:48.967 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:48.967 INFO:tasks.workunit.client.0.smithi125.stderr:+ ff=delete_pools_parallel 2023-10-12T02:14:48.967 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=84353 2023-10-12T02:14:48.967 INFO:tasks.workunit.client.0.smithi125.stderr:+ echo 'test delete_pools_parallel on pid 84353' 2023-10-12T02:14:48.968 INFO:tasks.workunit.client.0.smithi125.stdout:test delete_pools_parallel on pid 84353 2023-10-12T02:14:49.019 INFO:tasks.workunit.client.0.smithi125.stderr:+ pids[$f]=84353 2023-10-12T02:14:49.019 INFO:tasks.workunit.client.0.smithi125.stderr:+ ret=0 2023-10-12T02:14:49.020 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 1 -eq 1 ']' 2023-10-12T02:14:49.020 INFO:tasks.workunit.client.0.smithi125.stderr:+ for t in "${!pids[@]}" 2023-10-12T02:14:49.020 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83753 2023-10-12T02:14:49.021 INFO:tasks.workunit.client.0.smithi125.stderr:+ wait 83753 2023-10-12T02:14:49.021 INFO:tasks.workunit.client.0.smithi125.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2023-10-12T02:14:49.021 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.022 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:49.022 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.023 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:49.023 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:49.023 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:49.024 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:49.024 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_c_write_operations 2023-10-12T02:14:49.025 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:49.025 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.025 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:49.026 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -f /etc/bashrc ']' 2023-10-12T02:14:49.026 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.027 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:49.027 INFO:tasks.workunit.client.0.smithi125.stderr:+ . /etc/bashrc 2023-10-12T02:14:49.027 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:49.028 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.028 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:49.028 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.028 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -z '' ']' 2023-10-12T02:14:49.029 INFO:tasks.workunit.client.0.smithi125.stderr:++ BASHRCSOURCED=Y 2023-10-12T02:14:49.029 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.029 INFO:tasks.workunit.client.0.smithi125.stderr:++ shopt -q login_shell 2023-10-12T02:14:49.030 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:49.030 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:49.031 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.031 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:49.031 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.031 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:49.032 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:49.032 INFO:tasks.workunit.client.0.smithi125.stderr:+++ umask 2023-10-12T02:14:49.032 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' 0022 -eq 0 ']' 2023-10-12T02:14:49.032 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:49.033 INFO:tasks.workunit.client.0.smithi125.stderr:++ SHELL=/bin/bash 2023-10-12T02:14:49.033 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:49.033 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:49.034 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.034 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:49.034 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.035 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:49.035 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:49.035 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.036 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.036 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2023-10-12T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/bash_completion.sh 2023-10-12T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:49.040 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:49.040 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:49.040 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:49.041 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2023-10-12T02:14:49.041 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.041 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.042 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:49.042 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2023-10-12T02:14:49.042 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.042 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/cobbler.sh 2023-10-12T02:14:49.043 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.043 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:49.043 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:49.043 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:49.044 INFO:tasks.workunit.client.0.smithi125.stderr:+++ COBBLER_SERVER=172.21.0.11 2023-10-12T02:14:49.044 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.044 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2023-10-12T02:14:49.044 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.045 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.045 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorgrep.sh 2023-10-12T02:14:49.045 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:49.045 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.046 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:49.046 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:49.046 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:49.046 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:49.047 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.047 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.047 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:49.048 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:49.048 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.048 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:49.049 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:49.049 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:49.049 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:49.050 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.050 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:49.050 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.051 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:49.051 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.051 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:49.051 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:49.052 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.052 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:49.052 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.053 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:49.053 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:49.053 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:49.054 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:49.054 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.054 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:49.054 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.055 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:49.055 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:49.055 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:49.055 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84263/exe 2023-10-12T02:14:49.055 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:49.056 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.056 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:49.056 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.056 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:49.056 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:49.057 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:49.057 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:49.057 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:49.057 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:49.057 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:49.057 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:49.058 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:49.058 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:49.058 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:49.059 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:49.059 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:49.059 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.060 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:49.060 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:49.060 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:49.061 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:49.061 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.061 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:49.061 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.062 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:49.062 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:49.062 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.062 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.062 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:49.063 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.063 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:49.063 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.063 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:49.063 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.064 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:49.064 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:49.064 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:49.064 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.064 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:49.064 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.065 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:49.065 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:49.065 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2023-10-12T02:14:49.065 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:49.065 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'grep=grep --color=auto' 2023-10-12T02:14:49.066 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'egrep=egrep --color=auto' 2023-10-12T02:14:49.066 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_cls_remote_reads 2023-10-12T02:14:49.066 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'fgrep=fgrep --color=auto' 2023-10-12T02:14:49.066 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.067 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2023-10-12T02:14:49.067 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.067 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorls.sh 2023-10-12T02:14:49.067 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' '!' -t 0 ']' 2023-10-12T02:14:49.067 INFO:tasks.workunit.client.0.smithi125.stderr:+++ return 2023-10-12T02:14:49.068 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.068 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2023-10-12T02:14:49.068 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.068 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorsysstat.sh 2023-10-12T02:14:49.068 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export S_COLORS=auto 2023-10-12T02:14:49.069 INFO:tasks.workunit.client.0.smithi125.stderr:+++ S_COLORS=auto 2023-10-12T02:14:49.069 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.069 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2023-10-12T02:14:49.069 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.070 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2023-10-12T02:14:49.070 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorxzgrep.sh 2023-10-12T02:14:49.070 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:49.070 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:49.071 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:49.071 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:49.071 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.071 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:49.071 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.072 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:49.072 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:49.072 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:49.072 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:49.072 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:49.073 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:49.073 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.073 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:49.073 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.073 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:49.073 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:49.074 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.074 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.074 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:49.075 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.075 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:49.075 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.075 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:49.075 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:49.076 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.076 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:49.076 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.076 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:49.076 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:49.076 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84300/exe 2023-10-12T02:14:49.077 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:49.077 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:49.077 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84198/exe 2023-10-12T02:14:49.077 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:49.077 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:49.078 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:49.078 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:49.078 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:49.078 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:49.078 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.079 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:49.079 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:49.079 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:49.079 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:49.079 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.080 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:49.080 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:49.080 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:49.080 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.080 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:49.081 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.081 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:49.081 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.081 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:49.081 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.082 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:49.082 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:49.082 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2023-10-12T02:14:49.082 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2023-10-12T02:14:49.082 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2023-10-12T02:14:49.083 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.083 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2023-10-12T02:14:49.083 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:49.083 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.084 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:49.084 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:49.084 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/colorzgrep.sh 2023-10-12T02:14:49.084 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2023-10-12T02:14:49.085 INFO:tasks.workunit.client.0.smithi125.stderr:+++ /usr/libexec/grepconf.sh -c 2023-10-12T02:14:49.085 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:49.085 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:49.085 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:49.085 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:49.086 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:49.086 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:49.086 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:49.086 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:49.087 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:49.087 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:49.087 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_list_parallel.log 2023-10-12T02:14:49.088 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:49.088 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2023-10-12T02:14:49.088 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_c_read_operations 2023-10-12T02:14:49.089 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ list_parallel: /' 2023-10-12T02:14:49.089 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zgrep=zgrep --color=auto' 2023-10-12T02:14:49.089 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2023-10-12T02:14:49.090 INFO:tasks.workunit.client.0.smithi125.stderr:+++ alias 'zegrep=zegrep --color=auto' 2023-10-12T02:14:49.090 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.090 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2023-10-12T02:14:49.091 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.091 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/debuginfod.sh 2023-10-12T02:14:49.092 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2023-10-12T02:14:49.092 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.092 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2023-10-12T02:14:49.093 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.093 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/gawk.sh 2023-10-12T02:14:49.093 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.094 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2023-10-12T02:14:49.094 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.094 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/lang.sh 2023-10-12T02:14:49.094 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup 2023-10-12T02:14:49.095 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.095 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:49.095 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.095 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:49.095 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:49.096 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/locale 2023-10-12T02:14:49.096 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:49.096 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_list_parallel 2023-10-12T02:14:49.096 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:49.096 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:49.097 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:49.097 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.097 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:49.097 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.097 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:49.098 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:49.098 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.098 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.098 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:49.098 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:49.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:49.099 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:49.099 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.100 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:49.100 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.100 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:49.100 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:49.100 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.100 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG_backup=en_US.UTF-8 2023-10-12T02:14:49.101 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.101 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /etc/locale.conf ']' 2023-10-12T02:14:49.101 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -x /usr/bin/sed ']' 2023-10-12T02:14:49.101 INFO:tasks.workunit.client.0.smithi125.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2023-10-12T02:14:49.101 INFO:tasks.workunit.client.0.smithi125.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2023-10-12T02:14:49.102 INFO:tasks.workunit.client.0.smithi125.stderr:++++ export LANG=en_US.UTF-8 2023-10-12T02:14:49.102 INFO:tasks.workunit.client.0.smithi125.stderr:++++ LANG=en_US.UTF-8 2023-10-12T02:14:49.102 INFO:tasks.workunit.client.0.smithi125.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2023-10-12T02:14:49.102 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2023-10-12T02:14:49.102 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.103 INFO:tasks.workunit.client.0.smithi125.stderr:+++ LANG=en_US.UTF-8 2023-10-12T02:14:49.103 INFO:tasks.workunit.client.0.smithi125.stderr:+++ unset LANG_backup config 2023-10-12T02:14:49.103 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n '' ']' 2023-10-12T02:14:49.103 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -n en_US.UTF-8 ']' 2023-10-12T02:14:49.103 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' dumb = linux ']' 2023-10-12T02:14:49.104 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.104 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2023-10-12T02:14:49.104 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.104 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/less.sh 2023-10-12T02:14:49.104 INFO:tasks.workunit.client.0.smithi125.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2023-10-12T02:14:49.104 INFO:tasks.workunit.client.0.smithi125.stderr:++ for i in /etc/profile.d/*.sh 2023-10-12T02:14:49.105 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2023-10-12T02:14:49.105 INFO:tasks.workunit.client.0.smithi125.stderr:++ '[' '' ']' 2023-10-12T02:14:49.105 INFO:tasks.workunit.client.0.smithi125.stderr:++ . /etc/profile.d/which2.sh 2023-10-12T02:14:49.105 INFO:tasks.workunit.client.0.smithi125.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2023-10-12T02:14:49.105 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ api_c_read_operations: /' 2023-10-12T02:14:49.106 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84353/exe 2023-10-12T02:14:49.106 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:49.106 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:49.106 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:49.106 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:49.107 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:49.107 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:49.107 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:49.107 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:49.107 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:49.108 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:49.108 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:49.108 INFO:tasks.workunit.client.0.smithi125.stderr:+++++ readlink /proc/84331/exe 2023-10-12T02:14:49.108 INFO:tasks.workunit.client.0.smithi125.stderr:++++ basename /usr/bin/bash 2023-10-12T02:14:49.108 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_declare='declare -f' 2023-10-12T02:14:49.108 INFO:tasks.workunit.client.0.smithi125.stderr:+++ which_opt=-f 2023-10-12T02:14:49.109 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export which_declare 2023-10-12T02:14:49.109 INFO:tasks.workunit.client.0.smithi125.stderr:+++ export -f which 2023-10-12T02:14:49.109 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset i 2023-10-12T02:14:49.109 INFO:tasks.workunit.client.0.smithi125.stderr:++ unset -f pathmunge 2023-10-12T02:14:49.109 INFO:tasks.workunit.client.0.smithi125.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2023-10-12T02:14:49.110 INFO:tasks.workunit.client.0.smithi125.stderr:+ export PATH 2023-10-12T02:14:49.110 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2023-10-12T02:14:49.110 INFO:tasks.workunit.client.0.smithi125.stderr:+ unset rc 2023-10-12T02:14:49.110 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_delete_pools_parallel 2023-10-12T02:14:49.110 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2023-10-12T02:14:49.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ delete_pools_parallel: /' 2023-10-12T02:14:49.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_open_pools_parallel 2023-10-12T02:14:49.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ sed 's/^/ open_pools_parallel: /' 2023-10-12T02:14:49.111 INFO:tasks.workunit.client.0.smithi125.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2023-10-12T02:14:49.112 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2023-10-12T02:14:49.112 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2023-10-12T02:14:49.112 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2023-10-12T02:14:49.112 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2023-10-12T02:14:49.112 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/cls_remote_reads.cc:13: Skipped 2023-10-12T02:14:49.113 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: Not supported by crimson yet. Skipped 2023-10-12T02:14:49.113 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [ SKIPPED ] ClsTestRemoteReads.TestGather (0 ms) 2023-10-12T02:14:49.113 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (0 ms total) 2023-10-12T02:14:49.113 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: 2023-10-12T02:14:49.113 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2023-10-12T02:14:49.114 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (0 ms total) 2023-10-12T02:14:49.114 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [ PASSED ] 0 tests. 2023-10-12T02:14:49.114 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [ SKIPPED ] 1 test, listed below: 2023-10-12T02:14:49.114 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: [ SKIPPED ] ClsTestRemoteReads.TestGather 2023-10-12T02:14:49.114 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: 2023-10-12T02:14:48.431+0000 7f8a92631640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:49.114 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: 2023-10-12T02:14:48.432+0000 7f8a92631640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:49.115 INFO:tasks.workunit.client.0.smithi125.stdout: api_cls_remote_reads: 2023-10-12T02:14:48.432+0000 7f8a92631640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:52.719 INFO:tasks.workunit.client.0.smithi125.stdout:48.201+0000 7f40c6195640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f40b4036220 0x7f40b4038730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:14:52.720 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.201+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f40b802f040 con 0x56380ee737b0 2023-10-12T02:14:52.720 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.201+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40b802a7f0 con 0x56380ee737b0 2023-10-12T02:14:52.720 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.202+0000 7f40c8199640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f40b4036220 0x7f40b4038730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-12T02:14:52.720 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.202+0000 7f40c8199640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f40b4036220 0x7f40b4038730 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:14:52.721 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.224+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f40b80336b0 con 0x56380ee737b0 2023-10-12T02:14:52.721 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.224+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x56380ee0bf90 con 0x56380ee737b0 2023-10-12T02:14:52.721 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.767+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v15) v1 ==== 43+0+0 (crc 0 0 0) 0x7f40b8033480 con 0x56380ee737b0 2023-10-12T02:14:52.721 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.767+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f40b403ab40 con 0x56380ee737b0 2023-10-12T02:14:52.721 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.770+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 9256+0+0 (crc 0 0 0) 0x7f40b8032290 con 0x56380ee737b0 2023-10-12T02:14:52.722 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.770+0000 7f40cf853640 1 --2- 172.21.15.125:0/1696364954 >> v2:172.21.15.158:6804/3388116905 conn(0x56380ef0a2c0 0x56380efa31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T02:14:52.722 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.770+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:2 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380eef5450 con 0x56380ef0a2c0 2023-10-12T02:14:52.722 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.774+0000 7f40c919b640 1 --2- 172.21.15.125:0/1696364954 >> v2:172.21.15.158:6804/3388116905 conn(0x56380ef0a2c0 0x56380efa31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-12T02:14:52.722 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:48.794+0000 7f40c919b640 1 --2- 172.21.15.125:0/1696364954 >> v2:172.21.15.158:6804/3388116905 conn(0x56380ef0a2c0 0x56380efa31d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-12T02:14:52.723 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2023-10-12T02:14:52.791 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.718+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 1 ==== osd_op_reply(2 exist [writefull 0~ api_tier_pp: [==========] Running 77 tests from 4 test suites. 2023-10-12T02:14:52.791 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] Global test environment set-up. 2023-10-12T02:14:52.792 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2023-10-12T02:14:52.792 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: seed 83899 2023-10-12T02:14:52.792 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: 2023-10-12T02:14:48.246+0000 7f9051648940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:52.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: 2023-10-12T02:14:48.246+0000 7f9051648940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:52.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: 2023-10-12T02:14:48.246+0000 7f9051648940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:14:52.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2023-10-12T02:14:52.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:287: Skipped 2023-10-12T02:14:52.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierPP.Dirty (0 ms) 2023-10-12T02:14:52.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2023-10-12T02:14:52.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:2182: Skipped 2023-10-12T02:14:52.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierPP.FlushWriteRaces (0 ms) 2023-10-12T02:14:52.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2023-10-12T02:14:52.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:2561: Skipped 2023-10-12T02:14:52.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierPP.HitSetNone (0 ms) 2023-10-12T02:14:52.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (0 ms total) 2023-10-12T02:14:52.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: 2023-10-12T02:14:52.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2023-10-12T02:14:52.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2023-10-12T02:14:52.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Overlay (0 ms) 2023-10-12T02:14:52.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2023-10-12T02:14:52.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Promote (0 ms) 2023-10-12T02:14:52.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2023-10-12T02:14:52.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap (0 ms) 2023-10-12T02:14:52.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2023-10-12T02:14:52.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub (0 ms) 2023-10-12T02:14:52.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2023-10-12T02:14:52.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (0 ms) 2023-10-12T02:14:52.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2023-10-12T02:14:52.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.810 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Whiteout (0 ms) 2023-10-12T02:14:52.810 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2023-10-12T02:14:52.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (0 ms) 2023-10-12T02:14:52.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2023-10-12T02:14:52.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Evict (0 ms) 2023-10-12T02:14:52.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2023-10-12T02:14:52.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap (0 ms) 2023-10-12T02:14:52.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2023-10-12T02:14:52.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 (0 ms) 2023-10-12T02:14:52.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2023-10-12T02:14:52.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ListSnap (0 ms) 2023-10-12T02:14:52.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2023-10-12T02:14:52.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (0 ms) 2023-10-12T02:14:52.820 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2023-10-12T02:14:52.820 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.820 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TryFlush (0 ms) 2023-10-12T02:14:52.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2023-10-12T02:14:52.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Flush (0 ms) 2023-10-12T02:14:52.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2023-10-12T02:14:52.823 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.823 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.823 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.823 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.823 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap (0 ms) 2023-10-12T02:14:52.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2023-10-12T02:14:52.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.825 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces (0 ms) 2023-10-12T02:14:52.825 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2023-10-12T02:14:52.825 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.825 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.826 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.826 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.826 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace (0 ms) 2023-10-12T02:14:52.826 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2023-10-12T02:14:52.827 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.827 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.827 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.828 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.828 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead (0 ms) 2023-10-12T02:14:52.828 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2023-10-12T02:14:52.829 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.829 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.830 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.830 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.830 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite (0 ms) 2023-10-12T02:14:52.831 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2023-10-12T02:14:52.831 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.831 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.832 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.832 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.832 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim (0 ms) 2023-10-12T02:14:52.833 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2023-10-12T02:14:52.833 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.833 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.834 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.834 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead (0 ms) 2023-10-12T02:14:52.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2023-10-12T02:14:52.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead (0 ms) 2023-10-12T02:14:52.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2023-10-12T02:14:52.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.CachePin (0 ms) 2023-10-12T02:14:52.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2023-10-12T02:14:52.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead (0 ms) 2023-10-12T02:14:52.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2023-10-12T02:14:52.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead (0 ms) 2023-10-12T02:14:52.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2023-10-12T02:14:52.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead (0 ms) 2023-10-12T02:14:52.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2023-10-12T02:14:52.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset (0 ms) 2023-10-12T02:14:52.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2023-10-12T02:14:52.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead (1 ms) 2023-10-12T02:14:52.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2023-10-12T02:14:52.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount (0 ms) 2023-10-12T02:14:52.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2023-10-12T02:14:52.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (0 ms) 2023-10-12T02:14:52.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2023-10-12T02:14:52.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (0 ms) 2023-10-12T02:14:52.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2023-10-12T02:14:52.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (0 ms) 2023-10-12T02:14:52.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2023-10-12T02:14:52.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (0 ms) 2023-10-12T02:14:52.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2023-10-12T02:14:52.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.852 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.852 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.852 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount (0 ms) 2023-10-12T02:14:52.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2023-10-12T02:14:52.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict (0 ms) 2023-10-12T02:14:52.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2023-10-12T02:14:52.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote (0 ms) 2023-10-12T02:14:52.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2023-10-12T02:14:52.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (0 ms) 2023-10-12T02:14:52.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2023-10-12T02:14:52.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead (0 ms) 2023-10-12T02:14:52.859 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2023-10-12T02:14:52.859 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.859 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.860 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.860 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.861 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap (0 ms) 2023-10-12T02:14:52.861 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2023-10-12T02:14:52.861 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.862 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.862 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.862 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.863 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount (0 ms) 2023-10-12T02:14:52.863 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2023-10-12T02:14:52.863 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.864 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.864 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.864 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.865 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush (0 ms) 2023-10-12T02:14:52.865 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2023-10-12T02:14:52.866 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.866 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.866 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.867 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.867 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (0 ms) 2023-10-12T02:14:52.867 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2023-10-12T02:14:52.867 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.868 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.868 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.868 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.868 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback (0 ms) 2023-10-12T02:14:52.868 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2023-10-12T02:14:52.869 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.869 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.869 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:52.869 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:52.869 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (0 ms) 2023-10-12T02:14:52.869 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2023-10-12T02:14:52.870 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:52.870 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.318 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/cent5] v17'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:53.319 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.718+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:3 5.1 5:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa01b0 con 0x56380ef0a2c0 2023-10-12T02:14:53.319 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.718+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:4 5.1 5:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa0a20 con 0x56380ef0a2c0 2023-10-12T02:14:53.319 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.731+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 2 ==== osd_op_reply(3 exist [read 0~5 out=5b] v17'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:53.319 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.755+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 3 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:53.320 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (37 ms) 2023-10-12T02:14:53.320 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2023-10-12T02:14:53.320 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.755+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:5 5.1 5:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa0e00 con 0x56380ef0a2c0 2023-10-12T02:14:53.320 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.755+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:6 5.1 5:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa3710 con 0x56380ef0a2c0 2023-10-12T02:14:53.320 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.800+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 4 ==== osd_op_reply(5 exist [read 0~5 out=5b] v17'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:53.321 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.805+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 5 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:53.321 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (50 ms) 2023-10-12T02:14:53.321 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2023-10-12T02:14:53.321 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.806+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:7 5.1 5:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa0560 con 0x56380ef0a2c0 2023-10-12T02:14:53.321 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:52.806+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:8 5.1 5:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efe45a0 con 0x56380ef0a2c0 2023-10-12T02:14:53.321 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:53.317+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 6 ==== osd_op_reply(7 exist [read 0~5 out=5b] v17'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:53.322 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:53.317+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 7 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:53.322 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (512 ms) 2023-10-12T02:14:53.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.Aos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:53.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback (0 ms) 2023-10-12T02:14:53.795 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2023-10-12T02:14:53.795 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:53.795 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.796 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:53.796 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.796 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError (0 ms) 2023-10-12T02:14:53.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2023-10-12T02:14:53.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:53.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:53.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn (0 ms) 2023-10-12T02:14:53.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2023-10-12T02:14:53.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:223: Skipped 2023-10-12T02:14:53.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:233: Skipped 2023-10-12T02:14:53.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (0 ms) 2023-10-12T02:14:53.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (1 ms total) 2023-10-12T02:14:53.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: 2023-10-12T02:14:53.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2023-10-12T02:14:53.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:373: Skipped 2023-10-12T02:14:53.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2023-10-12T02:14:53.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:387: Skipped 2023-10-12T02:14:53.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:400: Skipped 2023-10-12T02:14:53.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.Dirty (0 ms) 2023-10-12T02:14:53.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2023-10-12T02:14:53.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:387: Skipped 2023-10-12T02:14:53.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:400: Skipped 2023-10-12T02:14:53.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.FlushWriteRaces (0 ms) 2023-10-12T02:14:53.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2023-10-12T02:14:53.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:387: Skipped 2023-10-12T02:14:53.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:400: Skipped 2023-10-12T02:14:53.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.CallForcesPromote (0 ms) 2023-10-12T02:14:53.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2023-10-12T02:14:53.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:387: Skipped 2023-10-12T02:14:53.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:400: Skipped 2023-10-12T02:14:53.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.HitSetNone (0 ms) 2023-10-12T02:14:53.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:381: Skipped 2023-10-12T02:14:53.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (0 ms total) 2023-10-12T02:14:53.807 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: 2023-10-12T02:14:53.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2023-10-12T02:14:53.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6267: Skipped 2023-10-12T02:14:53.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2023-10-12T02:14:53.808 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Overlay (0 ms) 2023-10-12T02:14:53.809 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2023-10-12T02:14:53.810 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.810 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.810 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.810 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Promote (0 ms) 2023-10-12T02:14:53.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2023-10-12T02:14:53.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.811 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap (0 ms) 2023-10-12T02:14:53.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2023-10-12T02:14:53.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.812 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (0 ms) 2023-10-12T02:14:53.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2023-10-12T02:14:53.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.813 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout (0 ms) 2023-10-12T02:14:53.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2023-10-12T02:14:53.814 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Evict (0 ms) 2023-10-12T02:14:53.815 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2023-10-12T02:14:53.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.816 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap (0 ms) 2023-10-12T02:14:53.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2023-10-12T02:14:53.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.817 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush (0 ms) 2023-10-12T02:14:53.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2023-10-12T02:14:53.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.818 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush (0 ms) 2023-10-12T02:14:53.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2023-10-12T02:14:53.819 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.820 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.820 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Flush (0 ms) 2023-10-12T02:14:53.821 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2023-10-12T02:14:53.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.822 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.823 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.823 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap (0 ms) 2023-10-12T02:14:53.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2023-10-12T02:14:53.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.824 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.825 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.825 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.825 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (0 ms) 2023-10-12T02:14:53.826 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2023-10-12T02:14:53.826 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.826 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.827 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.827 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.828 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace (0 ms) 2023-10-12T02:14:53.828 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2023-10-12T02:14:53.828 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.829 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.829 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.829 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.830 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead (0 ms) 2023-10-12T02:14:53.830 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2023-10-12T02:14:53.831 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.831 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.831 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.831 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.832 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim (0 ms) 2023-10-12T02:14:53.832 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2023-10-12T02:14:53.832 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.833 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.833 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.833 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.833 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (0 ms) 2023-10-12T02:14:53.834 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2023-10-12T02:14:53.834 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.834 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.834 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead (0 ms) 2023-10-12T02:14:53.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2023-10-12T02:14:53.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.835 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.CachePin (0 ms) 2023-10-12T02:14:53.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2023-10-12T02:14:53.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.836 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead (0 ms) 2023-10-12T02:14:53.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2023-10-12T02:14:53.837 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.838 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead (0 ms) 2023-10-12T02:14:53.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2023-10-12T02:14:53.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.839 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead (0 ms) 2023-10-12T02:14:53.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2023-10-12T02:14:53.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6278: Skipped 2023-10-12T02:14:53.840 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6288: Skipped 2023-10-12T02:14:53.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier (0 ms) 2023-10-12T02:14:53.841 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/tier_cxx.cc:6272: Skipped 2023-10-12T02:14:53.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: Not supported by crimson yet. Skipped 2023-10-12T02:14:53.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (0 ms total) 2023-10-12T02:14:53.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: 2023-10-12T02:14:53.842 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [----------] Global test environment tear-down 2023-10-12T02:14:53.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (5545 ms total) 2023-10-12T02:14:53.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ PASSED ] 0 tests. 2023-10-12T02:14:53.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] 77 tests, listed below: 2023-10-12T02:14:53.843 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierPP.Dirty 2023-10-12T02:14:53.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierPP.FlushWriteRaces 2023-10-12T02:14:53.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierPP.HitSetNone 2023-10-12T02:14:53.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Overlay 2023-10-12T02:14:53.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Promote 2023-10-12T02:14:53.844 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap 2023-10-12T02:14:53.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2023-10-12T02:14:53.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2023-10-12T02:14:53.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Whiteout 2023-10-12T02:14:53.845 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2023-10-12T02:14:53.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Evict 2023-10-12T02:14:53.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap 2023-10-12T02:14:53.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 2023-10-12T02:14:53.846 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ListSnap 2023-10-12T02:14:53.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2023-10-12T02:14:53.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TryFlush 2023-10-12T02:14:53.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.Flush 2023-10-12T02:14:53.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap 2023-10-12T02:14:53.847 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2023-10-12T02:14:53.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace 2023-10-12T02:14:53.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead 2023-10-12T02:14:53.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite 2023-10-12T02:14:53.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim 2023-10-12T02:14:53.848 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2023-10-12T02:14:53.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead 2023-10-12T02:14:53.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.CachePin 2023-10-12T02:14:53.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead 2023-10-12T02:14:53.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead 2023-10-12T02:14:53.849 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead 2023-10-12T02:14:53.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset 2023-10-12T02:14:53.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2023-10-12T02:14:53.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2023-10-12T02:14:53.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2023-10-12T02:14:53.850 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2023-10-12T02:14:53.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2023-10-12T02:14:53.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2023-10-12T02:14:53.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2023-10-12T02:14:53.851 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict 2023-10-12T02:14:53.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote 2023-10-12T02:14:53.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2023-10-12T02:14:53.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead 2023-10-12T02:14:53.853 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap 2023-10-12T02:14:53.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2023-10-12T02:14:53.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2023-10-12T02:14:53.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2023-10-12T02:14:53.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback 2023-10-12T02:14:53.854 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2023-10-12T02:14:53.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback 2023-10-12T02:14:53.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError 2023-10-12T02:14:53.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn 2023-10-12T02:14:53.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2023-10-12T02:14:53.855 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.Dirty 2023-10-12T02:14:53.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.FlushWriteRaces 2023-10-12T02:14:53.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.CallForcesPromote 2023-10-12T02:14:53.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTierECPP.HitSetNone 2023-10-12T02:14:53.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Overlay 2023-10-12T02:14:53.856 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Promote 2023-10-12T02:14:53.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap 2023-10-12T02:14:53.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2023-10-12T02:14:53.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout 2023-10-12T02:14:53.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Evict 2023-10-12T02:14:53.857 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap 2023-10-12T02:14:53.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush 2023-10-12T02:14:53.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush 2023-10-12T02:14:53.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.Flush 2023-10-12T02:14:53.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap 2023-10-12T02:14:53.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2023-10-12T02:14:53.858 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2023-10-12T02:14:53.859 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead 2023-10-12T02:14:53.859 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim 2023-10-12T02:14:53.860 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2023-10-12T02:14:53.860 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead 2023-10-12T02:14:53.860 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.CachePin 2023-10-12T02:14:53.860 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead 2023-10-12T02:14:53.860 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead 2023-10-12T02:14:53.861 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2023-10-12T02:14:53.861 INFO:tasks.workunit.client.0.smithi125.stdout: api_tier_pp: [ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier 2023-10-12T02:14:53.861 INFO:tasks.workunit.client.0.smithi125.stderr:+ for t in "${!pids[@]}" 2023-10-12T02:14:53.862 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83603 2023-10-12T02:14:53.862 INFO:tasks.workunit.client.0.smithi125.stderr:+ wait 83603 2023-10-12T02:14:56.512 INFO:tasks.workunit.client.0.smithi125.stdout:syncWriteCallback 2023-10-12T02:14:56.512 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:53.317+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:9 5.1 5:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa4a80 con 0x56380ef0a2c0 2023-10-12T02:14:56.512 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:55.401+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 8 ==== osd_op_reply(9 exist [write 0~5] v19'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:56.512 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (2084 ms) 2023-10-12T02:14:56.513 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2023-10-12T02:14:56.513 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:55.401+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:10 5.1 5:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa4a80 con 0x56380ef0a2c0 2023-10-12T02:14:56.513 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:55.688+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 9 ==== osd_op_reply(10 exist [write 0~5] v21'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:56.513 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (287 ms) 2023-10-12T02:14:56.513 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2023-10-12T02:14:56.514 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:55.688+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:11 5.1 5:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efc7340 con 0x56380ef0a2c0 2023-10-12T02:14:56.514 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.507+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 10 ==== osd_op_reply(11 exist [write 0~5] v21'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:56.514 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (819 ms) 2023-10-12T02:14:56.514 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2023-10-12T02:14:56.514 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.507+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:12 5.1 5:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa3a70 con 0x56380ef0a2c0 2023-10-12T02:14:56.515 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.507+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:13 5.1 5:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa43c0 con 0x56380ef0a2c0 2023-10-12T02:14:56.515 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.510+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 11 ==== osd_op_reply(12 exist [read 0~5 out=5b] v21'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:56.515 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.510+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 12 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:14:56.515 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (3 ms) 2023-10-12T02:14:56.516 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2023-10-12T02:14:56.516 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.510+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:14 5.1 5:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa3f20 con 0x56380ef0a2c0 2023-10-12T02:15:01.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-10-12T02:15:06.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-10-12T02:15:06.552 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~6s 2023-10-12T02:15:11.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2023-10-12T02:15:11.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~11s 2023-10-12T02:15:13.204 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.510+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:15 5.1 5:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efc7750 con 0x56380ef0a2c0 2023-10-12T02:15:13.204 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.516+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 13 ==== osd_op_reply(14 exist [read 0~5 out=5b] v21'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:15:13.205 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.519+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 14 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:15:13.205 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (9 ms) 2023-10-12T02:15:13.205 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2023-10-12T02:15:13.205 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.519+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:16 5.1 5:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa3f20 con 0x56380ef0a2c0 2023-10-12T02:15:13.206 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.519+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:17 5.1 5:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa4600 con 0x56380ef0a2c0 2023-10-12T02:15:13.206 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.535+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 15 ==== osd_op_reply(16 exist [read 0~5 out=5b] v21'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:15:13.206 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.535+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 16 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:15:13.206 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (16 ms) 2023-10-12T02:15:13.206 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2023-10-12T02:15:13.207 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:56.535+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:18 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa3f20 con 0x56380ef0a2c0 2023-10-12T02:15:13.207 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:57.532+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 17 ==== osd_op_reply(18 exist [writefull 0~5] v22'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f40bc00c430 con 0x56380ef0a2c0 2023-10-12T02:15:13.207 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (997 ms) 2023-10-12T02:15:13.207 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2023-10-12T02:15:13.207 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:57.533+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:19 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa3f20 con 0x56380ef0a2c0 2023-10-12T02:15:13.208 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:58.558+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 18 ==== osd_op_reply(19 exist [writefull 0~5] v23'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f40bc021040 con 0x56380ef0a2c0 2023-10-12T02:15:13.208 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1026 ms) 2023-10-12T02:15:13.208 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2023-10-12T02:15:13.208 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:14:58.559+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- osd_op(unknown.0.0:20 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x56380efa3f20 con 0x56380ef0a2c0 2023-10-12T02:15:15.908 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:13.203+0000 7f4 api_service: [==========] Running 4 tests from 1 test suite. 2023-10-12T02:15:15.909 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [----------] Global test environment set-up. 2023-10-12T02:15:15.909 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [----------] 4 tests from LibRadosService 2023-10-12T02:15:15.909 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2023-10-12T02:15:15.909 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:14:48.411+0000 7fd602eb57c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:15.910 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:14:48.412+0000 7fd602eb57c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:15.910 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:14:48.412+0000 7fd602eb57c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:15.910 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5151 ms) 2023-10-12T02:15:15.910 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2023-10-12T02:15:15.911 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ OK ] LibRadosService.RegisterLate (37 ms) 2023-10-12T02:15:15.911 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2023-10-12T02:15:15.911 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: cluster: 2023-10-12T02:15:15.911 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: id: bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:15:15.911 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: health: HEALTH_WARN 2023-10-12T02:15:15.912 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 5 pool(s) do not have an application enabled 2023-10-12T02:15:15.912 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.912 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: services: 2023-10-12T02:15:15.912 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: mon: 1 daemons, quorum a (age 83s) 2023-10-12T02:15:15.912 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: mgr: x(active, since 78s) 2023-10-12T02:15:15.912 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: osd: 4 osds: 4 up (since 76s), 4 in (since 80s) 2023-10-12T02:15:15.913 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: laundry: 2 daemons active (1 hosts) 2023-10-12T02:15:15.913 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.913 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: data: 2023-10-12T02:15:15.913 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: pools: 23 pools, 681 pgs 2023-10-12T02:15:15.914 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: objects: 2 objects, 577 KiB 2023-10-12T02:15:15.914 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: usage: 107 MiB used, 400 GiB / 400 GiB avail 2023-10-12T02:15:15.914 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: pgs: 98.678% pgs unknown 2023-10-12T02:15:15.914 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 672 unknown 2023-10-12T02:15:15.914 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 9 active+clean 2023-10-12T02:15:15.915 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.915 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.915 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: cluster: 2023-10-12T02:15:15.915 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: id: bd5090fa-2483-41ce-a0be-4321207eb2da 2023-10-12T02:15:15.915 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: health: HEALTH_WARN 2023-10-12T02:15:15.916 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 7 pool(s) do not have an application enabled 2023-10-12T02:15:15.916 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.916 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: services: 2023-10-12T02:15:15.916 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: mon: 1 daemons, quorum a (age 85s) 2023-10-12T02:15:15.917 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: mgr: x(active, since 80s) 2023-10-12T02:15:15.917 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: osd: 4 osds: 4 up (since 78s), 4 in (since 82s) 2023-10-12T02:15:15.917 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2023-10-12T02:15:15.917 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: laundry: 1 daemon active (1 hosts) 2023-10-12T02:15:15.918 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.918 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: data: 2023-10-12T02:15:15.918 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: pools: 28 pools, 841 pgs 2023-10-12T02:15:15.918 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: objects: 9 objects, 577 KiB 2023-10-12T02:15:15.918 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: usage: 111 MiB used, 400 GiB / 400 GiB avail 2023-10-12T02:15:15.919 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: pgs: 55.886% pgs unknown 2023-10-12T02:15:15.919 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 12.961% pgs not active 2023-10-12T02:15:15.919 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 470 unknown 2023-10-12T02:15:15.919 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 262 active+clean 2023-10-12T02:15:15.919 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 90 creating+peering 2023-10-12T02:15:15.919 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 19 creating+activating 2023-10-12T02:15:15.920 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.920 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: io: 2023-10-12T02:15:15.920 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: client: 1.2 KiB/s wr, 0 op/s rd, 1 op/s wr 2023-10-12T02:15:15.920 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.920 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.921 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ OK ] LibRadosService.StatusFormat (2262 ms) 2023-10-12T02:15:15.921 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ RUN ] LibRadosService.Status 2023-10-12T02:15:15.921 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ OK ] LibRadosService.Status (20045 ms) 2023-10-12T02:15:15.921 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [----------] 4 tests from LibRadosService (27495 ms total) 2023-10-12T02:15:15.921 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: 2023-10-12T02:15:15.922 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [----------] Global test environment tear-down 2023-10-12T02:15:15.922 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [==========] 4 tests from 1 test suite ran. (27495 ms total) 2023-10-12T02:15:15.922 INFO:tasks.workunit.client.0.smithi125.stdout: api_service: [ PASSED ] 4 tests. 2023-10-12T02:15:17.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~11s 2023-10-12T02:15:17.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~16s 2023-10-12T02:15:17.901 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: Running main() from gmock_main.cc 2023-10-12T02:15:17.901 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2023-10-12T02:15:17.902 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [----------] Global test environment set-up. 2023-10-12T02:15:17.902 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [----------] 9 tests from LibRadosPools 2023-10-12T02:15:17.902 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2023-10-12T02:15:17.902 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolList (2520 ms) 2023-10-12T02:15:17.903 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2023-10-12T02:15:17.903 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3007 ms) 2023-10-12T02:15:17.903 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2023-10-12T02:15:17.903 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3011 ms) 2023-10-12T02:15:17.904 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2023-10-12T02:15:17.904 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3012 ms) 2023-10-12T02:15:17.904 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2023-10-12T02:15:17.904 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3022 ms) 2023-10-12T02:15:17.905 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2023-10-12T02:15:17.905 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5015 ms) 2023-10-12T02:15:17.905 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2023-10-12T02:15:17.905 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5015 ms) 2023-10-12T02:15:17.905 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2023-10-12T02:15:17.906 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5032 ms) 2023-10-12T02:15:17.906 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2023-10-12T02:15:17.906 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/pool.cc:139: Skipped 2023-10-12T02:15:17.906 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: Not supported by crimson yet. Skipped 2023-10-12T02:15:17.907 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ SKIPPED ] LibRadosPools.PoolGetBaseTier (0 ms) 2023-10-12T02:15:17.907 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [----------] 9 tests from LibRadosPools (29634 ms total) 2023-10-12T02:15:17.908 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: 2023-10-12T02:15:17.908 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [----------] Global test environment tear-down 2023-10-12T02:15:17.908 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (29634 ms total) 2023-10-12T02:15:17.908 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ PASSED ] 8 tests. 2023-10-12T02:15:17.908 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ SKIPPED ] 1 test, listed below: 2023-10-12T02:15:17.909 INFO:tasks.workunit.client.0.smithi125.stdout: api_pool: [ SKIPPED ] LibRadosPools.PoolGetBaseTier 2023-10-12T02:15:18.737 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2023-10-12T02:15:18.738 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [----------] Global test environment set-up. 2023-10-12T02:15:18.738 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2023-10-12T02:15:18.738 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2023-10-12T02:15:18.739 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: 2023-10-12T02:14:48.410+0000 7f2849c697c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:18.739 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: 2023-10-12T02:14:48.410+0000 7f2849c697c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:18.739 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: 2023-10-12T02:14:48.410+0000 7f2849c697c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:18.739 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5152 ms) 2023-10-12T02:15:18.740 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2023-10-12T02:15:18.740 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (79 ms) 2023-10-12T02:15:18.740 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2023-10-12T02:15:18.740 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20036 ms) 2023-10-12T02:15:18.740 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2023-10-12T02:15:18.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: attempt 0 of 20 2023-10-12T02:15:18.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5056 ms) 2023-10-12T02:15:18.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30324 ms total) 2023-10-12T02:15:18.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: 2023-10-12T02:15:18.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [----------] Global test environment tear-down 2023-10-12T02:15:18.742 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30324 ms total) 2023-10-12T02:15:18.742 INFO:tasks.workunit.client.0.smithi125.stdout: api_service_pp: [ PASSED ] 4 tests. 2023-10-12T02:15:22.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~16s 2023-10-12T02:15:22.765 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~22s 2023-10-12T02:15:26.157 INFO:tasks.workunit.client.0.smithi125.stdout:0cabfe640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7f4090001dc0 con 0x56380ee737b0 2023-10-12T02:15:26.157 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:13.203+0000 7f40cabfe640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- ping magic: 0 v1 -- 0x7f4090002150 con 0x56380ef0a2c0 2023-10-12T02:15:26.157 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:13.203+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 9 ==== osd_map(16..39 src has 1..39) v4 ==== 27017+0+0 (crc 0 0 0) 0x7f40b8033440 con 0x56380ee737b0 2023-10-12T02:15:26.158 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:18.204+0000 7f40cabfe640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=40}) v3 -- 0x7f4090001dc0 con 0x56380ee737b0 2023-10-12T02:15:26.158 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:18.204+0000 7f40cabfe640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- ping magic: 0 v1 -- 0x7f4090002370 con 0x56380ef0a2c0 2023-10-12T02:15:26.158 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:18.204+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 10 ==== osd_map(40..44 src has 1..44) v4 ==== 2426+0+0 (crc 0 0 0) 0x7f40b8065040 con 0x56380ee737b0 2023-10-12T02:15:26.158 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:23.205+0000 7f40cabfe640 1 -- 172.21.15.125:0/1696364954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=45}) v3 -- 0x7f4090001dc0 con 0x56380ee737b0 2023-10-12T02:15:26.159 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:23.205+0000 7f40cabfe640 1 -- 172.21.15.125:0/1696364954 --> v2:172.21.15.158:6804/3388116905 -- ping magic: 0 v1 -- 0x7f40900025c0 con 0x56380ef0a2c0 2023-10-12T02:15:26.159 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:23.205+0000 7f40c6195640 1 -- 172.21.15.125:0/1696364954 <== mon.0 v2:172.21.15.125:3300/0 11 ==== osd_map(45..49 src has 1..49) v4 ==== 5398+0+0 (crc 0 0 0) 0x7f40b802d120 con 0x56380ee737b0 2023-10-12T02:15:26.159 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.155+0000 7f40c919b640 1 -- 172.21.15.125:0/1696364954 <== osd.3 v2:172.21.15.158:6804/3388116905 19 ==== osd_op_reply(20 exist [writefull 0~5] v24'7 uv0 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f40bc021040 con 0x56380ef0a2c0 2023-10-12T02:15:26.159 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (27596 ms) 2023-10-12T02:15:26.160 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.155+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 >> v2:172.21.15.158:6804/3388116905 conn(0x56380ef0a2c0 msgr2=0x56380efa31d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:15:26.160 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.155+0000 7f40cf853640 1 --2- 172.21.15.125:0/1696364954 >> v2:172.21.15.158:6804/3388116905 conn(0x56380ef0a2c0 0x56380efa31d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:15:26.160 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.155+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f40b4036220 msgr2=0x7f40b4038730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:15:26.160 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.155+0000 7f40cf853640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f40b4036220 0x7f40b4038730 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:15:26.161 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.155+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 msgr2=0x56380eef1930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-12T02:15:26.161 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.155+0000 7f40cf853640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380eef1930 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:15:26.161 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.156+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 shutdown_connections 2023-10-12T02:15:26.161 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.156+0000 7f40cf853640 1 --2- 172.21.15.125:0/1696364954 >> v2:172.21.15.158:6804/3388116905 conn(0x56380ef0a2c0 0x56380efa31d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:15:26.162 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.156+0000 7f40cf853640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.158:6800/1559993426,v1:172.21.15.158:6801/1559993426] conn(0x7f40b4036220 0x7f40b4038730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:15:26.162 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.156+0000 7f40cf853640 1 --2- 172.21.15.125:0/1696364954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380ee737b0 0x56380eef1930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T02:15:26.162 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.156+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 >> 172.21.15.125:0/1696364954 conn(0x56380ed7cd80 msgr2=0x56380ef07cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-12T02:15:26.163 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.156+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 shutdown_connections 2023-10-12T02:15:26.163 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.156+0000 7f40cf853640 1 -- 172.21.15.125:0/1696364954 wait complete. 2023-10-12T02:15:26.163 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [----------] 12 tests from AsioRados (33437 ms total) 2023-10-12T02:15:26.164 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: 2023-10-12T02:15:26.164 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [----------] Global test environment tear-down 2023-10-12T02:15:26.164 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (37992 ms total) 2023-10-12T02:15:26.164 INFO:tasks.workunit.client.0.smithi125.stdout: api_asio: [ PASSED ] 12 tests. 2023-10-12T02:15:26.165 INFO:tasks.workunit.client.0.smithi125.stderr:+ for t in "${!pids[@]}" 2023-10-12T02:15:26.165 INFO:tasks.workunit.client.0.smithi125.stderr:+ pid=83844 2023-10-12T02:15:26.165 INFO:tasks.workunit.client.0.smithi125.stderr:+ wait 83844 2023-10-12T02:15:26.792 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2023-10-12T02:15:26.792 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [----------] Global test environment set-up. 2023-10-12T02:15:26.792 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2023-10-12T02:15:26.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2023-10-12T02:15:26.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2023-10-12T02:15:26.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2023-10-12T02:15:26.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: 2023-10-12T02:15:26.793 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2023-10-12T02:15:26.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2023-10-12T02:15:26.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: 2023-10-12T02:14:48.222+0000 7f0cf0437640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:26.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: 2023-10-12T02:14:48.223+0000 7f0cf0437640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:26.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: 2023-10-12T02:14:48.223+0000 7f0cf0437640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T02:15:26.794 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2023-10-12T02:15:26.795 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: 2023-10-12T02:14:48.400+0000 7f0cf0437640 -1 failed for service _ceph-mon._tcp 2023-10-12T02:15:26.795 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: 2023-10-12T02:14:48.400+0000 7f0cf0437640 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2023-10-12T02:15:26.795 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (182 ms) 2023-10-12T02:15:26.795 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2023-10-12T02:15:26.796 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2014 ms) 2023-10-12T02:15:26.796 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2196 ms total) 2023-10-12T02:15:26.796 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: 2023-10-12T02:15:26.796 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2023-10-12T02:15:26.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2023-10-12T02:15:26.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x7f0cd800a0e0 2023-10-12T02:15:26.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x7f0cd0005d20 2023-10-12T02:15:26.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef429a3d0 2023-10-12T02:15:26.797 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4293ec0 2023-10-12T02:15:26.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef424c0e0 2023-10-12T02:15:26.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4220be0 2023-10-12T02:15:26.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef421e020 2023-10-12T02:15:26.798 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef456f3d0 2023-10-12T02:15:26.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef456fe30 2023-10-12T02:15:26.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4564ab0 2023-10-12T02:15:26.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4563bf0 2023-10-12T02:15:26.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef456adf0 2023-10-12T02:15:26.799 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef456b850 2023-10-12T02:15:26.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef456c2b0 2023-10-12T02:15:26.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4571320 2023-10-12T02:15:26.800 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4563fe0 2023-10-12T02:15:26.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4572780 2023-10-12T02:15:26.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4573280 2023-10-12T02:15:26.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4573dd0 2023-10-12T02:15:26.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4574920 2023-10-12T02:15:26.801 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4575470 2023-10-12T02:15:26.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4575fc0 2023-10-12T02:15:26.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4576b10 2023-10-12T02:15:26.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4577660 2023-10-12T02:15:26.802 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef45781b0 2023-10-12T02:15:26.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4578d00 2023-10-12T02:15:26.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4579850 2023-10-12T02:15:26.803 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef457a3a0 2023-10-12T02:15:26.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef457aef0 2023-10-12T02:15:26.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef457ba40 2023-10-12T02:15:26.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef457c530 2023-10-12T02:15:26.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef457d020 2023-10-12T02:15:26.804 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef457db10 2023-10-12T02:15:26.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef457e660 2023-10-12T02:15:26.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 0x55aef4573a20 2023-10-12T02:15:26.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: started 35 aios 2023-10-12T02:15:26.805 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: waiting 0x7f0cd800a0e0 2023-10-12T02:15:26.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: waiting 0x7f0cd0005d20 2023-10-12T02:15:26.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: waiting 0x55aef429a3d0 2023-10-12T02:15:26.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: waiting 0x55aef4293ec0 2023-10-12T02:15:26.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: waiting 0x55aef424c0e0 2023-10-12T02:15:26.806 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: waiting 0x55aef4220be0 2023-10-12T02:15:28.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~22s 2023-10-12T02:15:28.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~27s 2023-10-12T02:15:28.603 INFO:tasks.workunit.client.0.smithi125.stdout: api_misc: waitin api_watch_notify_pp: Running main() from gmock_main.cc 2023-10-12T02:15:28.604 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2023-10-12T02:15:28.604 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2023-10-12T02:15:28.604 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2023-10-12T02:15:28.604 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:373: Skipped 2023-10-12T02:15:28.605 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: Not supported by crimson yet. Skipped 2023-10-12T02:15:28.605 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2023-10-12T02:15:28.605 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:387: Skipped 2023-10-12T02:15:28.605 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: Not supported by crimson yet. Skipped 2023-10-12T02:15:28.605 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:400: Skipped 2023-10-12T02:15:28.606 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: Not supported by crimson yet. Skipped 2023-10-12T02:15:28.606 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify (0 ms) 2023-10-12T02:15:28.606 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2023-10-12T02:15:28.606 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:387: Skipped 2023-10-12T02:15:28.607 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: Not supported by crimson yet. Skipped 2023-10-12T02:15:28.607 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:400: Skipped 2023-10-12T02:15:28.607 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: Not supported by crimson yet. Skipped 2023-10-12T02:15:28.607 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (0 ms) 2023-10-12T02:15:28.607 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6684-g9587ffdf/rpm/el9/BUILD/ceph-18.0.0-6684-g9587ffdf/src/test/librados/testcase_cxx.cc:381: Skipped 2023-10-12T02:15:28.608 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: Not supported by crimson yet. Skipped 2023-10-12T02:15:28.608 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (0 ms total) 2023-10-12T02:15:28.608 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: 2023-10-12T02:15:28.608 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2023-10-12T02:15:28.608 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2023-10-12T02:15:28.608 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: notify 2023-10-12T02:15:28.609 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (38737 ms) 2023-10-12T02:15:28.609 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2023-10-12T02:15:28.609 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: notify 2023-10-12T02:15:28.609 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (29 ms) 2023-10-12T02:15:28.609 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2023-10-12T02:15:28.610 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (20 ms) 2023-10-12T02:15:28.610 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2023-10-12T02:15:28.610 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (19 ms) 2023-10-12T02:15:28.610 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2023-10-12T02:15:33.573 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~27s 2023-10-12T02:15:33.574 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~33s 2023-10-12T02:15:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~32s 2023-10-12T02:15:38.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~38s 2023-10-12T02:15:42.029 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964904256 notify_id 231928233984 notifier_gid 4234 2023-10-12T02:15:42.029 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (22 ms) 2023-10-12T02:15:42.029 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2023-10-12T02:15:42.029 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 231928233985 notifier_gid 4234 2023-10-12T02:15:42.030 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (15 ms) 2023-10-12T02:15:42.030 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2023-10-12T02:15:42.030 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 231928233987 notifier_gid 4234 2023-10-12T02:15:42.031 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (18 ms) 2023-10-12T02:15:42.031 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2023-10-12T02:15:42.031 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 231928233988 notifier_gid 4234 2023-10-12T02:15:42.031 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (16 ms) 2023-10-12T02:15:42.031 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2023-10-12T02:15:42.032 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964904256 notify_id 231928233986 notifier_gid 4234 2023-10-12T02:15:42.032 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (15 ms) 2023-10-12T02:15:42.032 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2023-10-12T02:15:42.032 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 231928233989 notifier_gid 4234 2023-10-12T02:15:42.032 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (17 ms) 2023-10-12T02:15:42.033 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2023-10-12T02:15:42.033 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: trying... 2023-10-12T02:15:42.033 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 231928233987 notifier_gid 4234 2023-10-12T02:15:42.033 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: timed out 2023-10-12T02:15:42.033 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: flushing 2023-10-12T02:15:42.033 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: flushed 2023-10-12T02:15:42.034 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3011 ms) 2023-10-12T02:15:42.034 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2023-10-12T02:15:42.034 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: trying... 2023-10-12T02:15:42.034 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 244813135876 notifier_gid 4234 2023-10-12T02:15:42.034 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: timed out 2023-10-12T02:15:42.035 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: flushing 2023-10-12T02:15:42.035 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: flushed 2023-10-12T02:15:42.035 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3012 ms) 2023-10-12T02:15:42.035 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2023-10-12T02:15:42.035 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: List watches 2023-10-12T02:15:42.035 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: notify2 2023-10-12T02:15:42.036 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 257698037765 notifier_gid 4234 2023-10-12T02:15:42.036 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: notify2 done 2023-10-12T02:15:42.036 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: watch_check 2023-10-12T02:15:42.036 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: unwatch2 2023-10-12T02:15:42.036 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: flushing 2023-10-12T02:15:42.037 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: done 2023-10-12T02:15:42.037 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3018 ms) 2023-10-12T02:15:42.037 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2023-10-12T02:15:42.037 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: List watches 2023-10-12T02:15:42.037 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: notify2 2023-10-12T02:15:42.037 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: handle_notify cookie 93937964902208 notify_id 270582939654 notifier_gid 4234 2023-10-12T02:15:42.038 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: notify2 done 2023-10-12T02:15:42.038 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: watch_check 2023-10-12T02:15:42.038 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: unwatch2 2023-10-12T02:15:42.038 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: flushing 2023-10-12T02:15:42.038 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: done 2023-10-12T02:15:42.039 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3452 ms) 2023-10-12T02:15:42.039 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (51401 ms total) 2023-10-12T02:15:42.039 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: 2023-10-12T02:15:42.040 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2023-10-12T02:15:42.040 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (53709 ms total) 2023-10-12T02:15:42.040 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ PASSED ] 14 tests. 2023-10-12T02:15:42.040 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ SKIPPED ] 2 tests, listed below: 2023-10-12T02:15:42.040 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify 2023-10-12T02:15:42.041 INFO:tasks.workunit.client.0.smithi125.stdout: api_watch_notify_pp: [ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2023-10-12T02:15:44.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~38s 2023-10-12T02:15:44.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-10-12T02:15:44.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~43s 2023-10-12T02:15:49.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~43s 2023-10-12T02:15:49.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2023-10-12T02:15:49.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~49s 2023-10-12T02:15:54.889 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~48s 2023-10-12T02:15:54.890 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~11s 2023-10-12T02:15:54.890 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~54s 2023-10-12T02:16:00.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~54s 2023-10-12T02:16:00.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~16s 2023-10-12T02:16:00.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~59s 2023-10-12T02:16:05.497 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~59s 2023-10-12T02:16:05.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~21s 2023-10-12T02:16:05.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2023-10-12T02:16:10.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~64s 2023-10-12T02:16:10.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~27s 2023-10-12T02:16:10.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2023-10-12T02:16:16.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2023-10-12T02:16:16.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~32s 2023-10-12T02:16:16.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~75s 2023-10-12T02:16:21.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~75s 2023-10-12T02:16:21.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~37s 2023-10-12T02:16:21.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~80s 2023-10-12T02:16:26.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~80s 2023-10-12T02:16:26.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~42s 2023-10-12T02:16:26.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~86s 2023-10-12T02:16:32.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~86s 2023-10-12T02:16:32.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~48s 2023-10-12T02:16:32.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~91s 2023-10-12T02:16:37.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~91s 2023-10-12T02:16:37.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~53s 2023-10-12T02:16:37.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~96s 2023-10-12T02:16:40.750 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:16:40.748+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:16:42.624 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~96s 2023-10-12T02:16:42.624 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~58s 2023-10-12T02:16:42.625 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~102s 2023-10-12T02:16:45.750 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:16:45.749+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:16:47.928 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~101s 2023-10-12T02:16:47.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~64s 2023-10-12T02:16:47.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~107s 2023-10-12T02:16:50.751 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:16:50.750+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:16:53.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~107s 2023-10-12T02:16:53.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~69s 2023-10-12T02:16:53.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~112s 2023-10-12T02:16:55.753 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:16:55.751+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:16:58.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~112s 2023-10-12T02:16:58.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~74s 2023-10-12T02:16:58.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~118s 2023-10-12T02:17:00.754 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:00.753+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:03.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~117s 2023-10-12T02:17:03.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~80s 2023-10-12T02:17:03.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~123s 2023-10-12T02:17:05.755 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:05.754+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:09.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~123s 2023-10-12T02:17:09.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~85s 2023-10-12T02:17:09.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~128s 2023-10-12T02:17:10.756 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:10.754+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:14.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~128s 2023-10-12T02:17:14.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~90s 2023-10-12T02:17:14.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~134s 2023-10-12T02:17:15.757 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:15.756+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:19.753 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~133s 2023-10-12T02:17:19.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~95s 2023-10-12T02:17:19.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~139s 2023-10-12T02:17:20.758 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:20.757+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:25.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~139s 2023-10-12T02:17:25.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~101s 2023-10-12T02:17:25.058 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~144s 2023-10-12T02:17:25.759 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:25.757+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:30.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~144s 2023-10-12T02:17:30.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~106s 2023-10-12T02:17:30.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~149s 2023-10-12T02:17:30.760 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:30.759+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:35.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~149s 2023-10-12T02:17:35.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~111s 2023-10-12T02:17:35.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~155s 2023-10-12T02:17:35.762 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:35.761+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:40.765 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:40.762+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:40.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~155s 2023-10-12T02:17:40.970 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~117s 2023-10-12T02:17:40.970 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~160s 2023-10-12T02:17:45.764 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:45.763+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:46.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~160s 2023-10-12T02:17:46.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~122s 2023-10-12T02:17:46.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~165s 2023-10-12T02:17:50.765 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:50.764+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:51.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~165s 2023-10-12T02:17:51.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~127s 2023-10-12T02:17:51.579 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~171s 2023-10-12T02:17:55.767 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:17:55.765+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:17:56.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~170s 2023-10-12T02:17:56.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~133s 2023-10-12T02:17:56.883 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~176s 2023-10-12T02:18:00.768 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:00.766+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~176s 2023-10-12T02:18:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~138s 2023-10-12T02:18:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~181s 2023-10-12T02:18:05.768 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:05.767+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:07.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~181s 2023-10-12T02:18:07.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~143s 2023-10-12T02:18:07.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~187s 2023-10-12T02:18:10.769 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:10.768+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:12.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~186s 2023-10-12T02:18:12.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~149s 2023-10-12T02:18:12.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~192s 2023-10-12T02:18:15.771 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:15.770+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:18.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~192s 2023-10-12T02:18:18.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~154s 2023-10-12T02:18:18.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~197s 2023-10-12T02:18:20.772 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:20.770+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:23.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~197s 2023-10-12T02:18:23.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~159s 2023-10-12T02:18:23.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~202s 2023-10-12T02:18:25.773 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:25.771+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:28.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~202s 2023-10-12T02:18:28.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~164s 2023-10-12T02:18:28.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~208s 2023-10-12T02:18:30.774 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:30.772+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:34.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~208s 2023-10-12T02:18:34.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~170s 2023-10-12T02:18:34.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~213s 2023-10-12T02:18:35.813 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:35.774+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:39.312 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~213s 2023-10-12T02:18:39.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~175s 2023-10-12T02:18:39.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~218s 2023-10-12T02:18:40.776 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:40.775+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:44.616 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~218s 2023-10-12T02:18:44.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~180s 2023-10-12T02:18:44.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~224s 2023-10-12T02:18:45.777 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:45.776+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:49.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~223s 2023-10-12T02:18:49.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~186s 2023-10-12T02:18:49.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~229s 2023-10-12T02:18:50.778 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:50.777+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:18:55.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~229s 2023-10-12T02:18:55.224 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~191s 2023-10-12T02:18:55.224 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~234s 2023-10-12T02:18:55.779 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:18:55.778+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:00.527 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~234s 2023-10-12T02:19:00.527 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~196s 2023-10-12T02:19:00.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~240s 2023-10-12T02:19:00.780 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:00.779+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:05.781 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:05.780+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:05.831 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~239s 2023-10-12T02:19:05.831 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~202s 2023-10-12T02:19:05.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~245s 2023-10-12T02:19:10.782 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:10.781+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:11.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~245s 2023-10-12T02:19:11.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~207s 2023-10-12T02:19:11.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~250s 2023-10-12T02:19:15.784 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:15.781+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:16.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~250s 2023-10-12T02:19:16.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~212s 2023-10-12T02:19:16.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~255s 2023-10-12T02:19:20.785 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:20.782+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:21.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~255s 2023-10-12T02:19:21.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~217s 2023-10-12T02:19:21.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~261s 2023-10-12T02:19:25.786 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:25.784+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:27.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~261s 2023-10-12T02:19:27.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~223s 2023-10-12T02:19:27.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~266s 2023-10-12T02:19:30.786 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:30.785+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:32.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~266s 2023-10-12T02:19:32.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~228s 2023-10-12T02:19:32.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~271s 2023-10-12T02:19:35.788 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:35.787+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:37.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~271s 2023-10-12T02:19:37.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~233s 2023-10-12T02:19:37.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~277s 2023-10-12T02:19:40.789 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:40.788+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:42.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~277s 2023-10-12T02:19:42.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~239s 2023-10-12T02:19:42.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~282s 2023-10-12T02:19:45.790 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:45.789+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:48.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~282s 2023-10-12T02:19:48.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~244s 2023-10-12T02:19:48.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~287s 2023-10-12T02:19:50.791 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:50.790+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:53.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~287s 2023-10-12T02:19:53.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~249s 2023-10-12T02:19:53.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~293s 2023-10-12T02:19:55.792 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:19:55.791+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:19:58.869 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~292s 2023-10-12T02:19:58.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~255s 2023-10-12T02:19:58.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~298s 2023-10-12T02:20:00.793 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:20:00.792+0000 7f75c73f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 v2:172.21.15.125:6801/2325473832 for 29sec e72 v72) 2023-10-12T02:20:04.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~298s 2023-10-12T02:20:04.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~260s 2023-10-12T02:20:04.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~303s 2023-10-12T02:20:04.174 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-10-12T02:20:04.477 INFO:tasks.ceph.osd.3:Sent signal 15 2023-10-12T02:20:04.478 INFO:tasks.ceph.mon.a:Sent signal 15 2023-10-12T02:20:04.478 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-10-12T02:20:04.479 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:20:04.478+0000 7f75ca3fc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 81562) UID: 0 2023-10-12T02:20:04.479 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-12T02:20:04.478+0000 7f75ca3fc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-12T02:20:04.480 INFO:tasks.ceph.mgr.x.smithi158.stderr:2023-10-12T02:20:04.479+0000 7f77465f7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 81467) UID: 0 2023-10-12T02:55:40.740 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: Running main() from gmock_main.cc 2023-10-12T02:55:40.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2023-10-12T02:55:40.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [----------] Global test environment set-up. 2023-10-12T02:55:40.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [----------] 26 tests from LibRadosAio 2023-10-12T02:55:40.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2023-10-12T02:55:40.741 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ OK ] LibRadosAio.TooBig (2670 ms) 2023-10-12T02:55:40.742 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2023-10-12T02:55:40.742 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (36172 ms) 2023-10-12T02:55:40.742 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2023-10-12T02:55:40.742 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3004 ms) 2023-10-12T02:55:40.743 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2023-10-12T02:55:40.743 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3018 ms) 2023-10-12T02:55:40.743 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2023-10-12T02:55:40.743 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3023 ms) 2023-10-12T02:55:40.743 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2023-10-12T02:55:40.743 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3015 ms) 2023-10-12T02:55:40.744 INFO:tasks.workunit.client.0.smithi125.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2023-10-12T02:55:40.744 INFO:tasks.workunit.client.0.smithi125.stderr:bash: line 1: 83630 Alarm clock ceph_test_rados_api_aio 2>&1 2023-10-12T02:55:40.744 INFO:tasks.workunit.client.0.smithi125.stderr: 83631 Done | tee ceph_test_rados_api_aio.log 2023-10-12T02:55:40.745 INFO:tasks.workunit.client.0.smithi125.stderr: 83632 Done | sed "s/^/ api_aio: /" 2023-10-12T05:14:48.073 INFO:tasks.workunit.client.0.smithi125.stderr:++ cleanup 2023-10-12T05:14:48.093 INFO:tasks.workunit.client.0.smithi125.stderr:++ pkill -P 83556 2023-10-12T05:14:48.102 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-12T05:14:48.102 INFO:tasks.workunit.client.0.smithi125.stderr:++ true 2023-10-12T05:14:48.102 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2023-10-12T05:14:48.103 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-12T05:14:48.360 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi125 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2023-10-12T05:14:48.430 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7b33718969f94f738e77ad1dfecfdf5f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi125 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2023-10-12T05:14:48.433 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-12T05:14:48.443 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-12T05:14:48.444 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-12T05:14:48.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.559+0000 7f394fc06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-10-12T05:14:48.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.563+0000 7f394fc06640 1 Processor -- start 2023-10-12T05:14:48.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.563+0000 7f394fc06640 1 -- start start 2023-10-12T05:14:48.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.564+0000 7f394fc06640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f3948134910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:14:48.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.564+0000 7f394fc06640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f394812f5d0 2023-10-12T05:14:48.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.564+0000 7f394fc06640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f39481324f0 2023-10-12T05:14:48.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.564+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f3948134910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:48.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.564+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f394812f9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:48.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.564+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f3948134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:14:48.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.765+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f394812f9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:48.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.765+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f3948134910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:48.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:48.765+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f3948134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:14:49.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:49.166+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f394812f9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:49.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:49.166+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f3948134910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:49.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:49.166+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f3948134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:14:49.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:49.967+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f394812f9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:49.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:49.967+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f3948134910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:49.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:49.967+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f3948134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:14:51.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f3948134910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:14:51.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f3948134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:14:51.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f394812f9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:14:51.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:14:51.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f394812f5d0 2023-10-12T05:14:51.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f39481324f0 2023-10-12T05:14:51.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:51.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:51.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.565+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:14:51.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.766+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:51.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.766+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:51.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:51.766+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:14:52.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:52.167+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:52.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:52.167+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:52.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:52.167+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:14:52.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:52.968+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:52.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:52.968+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:52.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:52.968+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:14:54.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:14:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:14:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:14:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:14:54.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f394812f5d0 2023-10-12T05:14:54.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f39481324f0 2023-10-12T05:14:54.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:54.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:54.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.565+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:14:54.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.766+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:54.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.766+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:54.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:54.766+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:14:55.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:55.167+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:55.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:55.167+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:55.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:55.167+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:14:55.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:55.968+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:55.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:55.968+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:55.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:55.968+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:14:57.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.565+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 msgr2=0x7f392c008de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:14:57.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.565+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f39481324f0 0x7f392c008de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:14:57.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.565+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:14:57.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.565+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:14:57.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.566+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f394812f5d0 2023-10-12T05:14:57.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.566+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00b630 2023-10-12T05:14:57.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.566+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:57.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.566+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:57.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.566+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:14:57.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.767+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:57.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.767+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:57.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:57.767+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:14:58.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:58.168+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:58.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:58.168+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:58.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:58.168+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:14:58.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:58.969+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:14:58.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:58.969+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:14:58.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:14:58.969+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:00.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:00.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:00.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:00.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:00.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f394812f5d0 2023-10-12T05:15:00.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00b630 2023-10-12T05:15:00.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:00.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:00.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.566+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:00.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.767+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:00.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.767+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:00.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:00.767+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:01.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:01.168+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:01.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:01.168+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:01.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:01.168+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:01.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:01.969+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:01.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:01.969+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:01.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:01.969+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:03.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:03.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:03.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:03.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f394812f5d0 2023-10-12T05:15:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00b630 2023-10-12T05:15:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:03.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.566+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:03.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.767+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:03.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.767+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:03.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:03.767+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:04.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:04.168+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:04.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:04.168+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:04.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:04.168+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:04.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:04.969+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:04.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:04.969+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:04.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:04.969+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:06.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:06.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:06.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f394812f5d0 2023-10-12T05:15:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00b630 2023-10-12T05:15:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:06.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:06.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.567+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:06.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.768+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:06.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.768+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:06.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:06.768+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:07.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:07.169+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:07.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:07.169+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:07.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:07.169+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:07.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:07.970+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:07.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:07.970+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:07.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:07.970+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:09.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:09.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:09.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f394812f5d0 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:09.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:09.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c005610 2023-10-12T05:15:09.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00b630 2023-10-12T05:15:09.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c009530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:09.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:09.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.567+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:09.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.768+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c009530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:09.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.768+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:09.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:09.768+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:10.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:10.169+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c009530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:10.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:10.169+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:10.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:10.169+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:10.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:10.970+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c009530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:10.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:10.970+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:10.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:10.970+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:12.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:12.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c009530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c005610 2023-10-12T05:15:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00b630 2023-10-12T05:15:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.567+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:12.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.768+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:12.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.768+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:12.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:12.768+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:13.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:13.169+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:13.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:13.169+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:13.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:13.169+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:13.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:13.970+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:13.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:13.970+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:13.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:13.970+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:15.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:15.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00b630 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:15.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:15.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c009940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:15.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c005610 2023-10-12T05:15:15.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c009530 2023-10-12T05:15:15.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:15.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:15.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.568+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:15.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.769+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:15.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.769+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:15.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:15.769+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:16.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:16.170+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:16.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:16.170+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:16.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:16.170+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:16.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:16.971+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:16.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:16.971+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:16.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:16.971+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:18.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c009940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:18.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:18.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:18.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:18.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c005610 2023-10-12T05:15:18.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c009530 2023-10-12T05:15:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.568+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:18.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.769+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:18.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.769+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:18.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:18.769+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:19.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:19.170+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:19.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:19.170+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:19.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:19.170+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:19.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:19.971+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:19.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:19.971+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:19.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:19.971+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:21.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:21.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:21.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005610 legacy=0x7f392c00bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:21.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:21.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c009940 2023-10-12T05:15:21.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c009530 2023-10-12T05:15:21.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c009940 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:21.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:21.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.568+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:21.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.769+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c009940 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:21.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.769+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:21.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:21.769+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:22.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:22.170+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c009940 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:22.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:22.170+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:22.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:22.170+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:22.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:22.971+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:22.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:22.971+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c009940 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:22.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:22.971+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:24.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:24.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:24.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c009940 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:24.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:24.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c00bfb0 2023-10-12T05:15:24.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c009530 2023-10-12T05:15:24.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:24.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:24.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.569+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:24.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.770+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:24.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.770+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:24.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:24.770+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:25.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:25.171+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:25.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:25.171+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:25.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:25.171+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:25.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:25.972+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:25.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:25.972+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:25.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:25.972+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:27.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:27.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:27.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:27.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:27.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c00bfb0 2023-10-12T05:15:27.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c009530 2023-10-12T05:15:27.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:27.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:27.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.569+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:27.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.770+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:27.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.770+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:27.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:27.770+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:28.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:28.171+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:28.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:28.171+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:28.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:28.171+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:28.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:28.972+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:28.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:28.972+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:28.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:28.972+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c009530 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c006450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c00bfb0 2023-10-12T05:15:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.569+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c006450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:30.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.770+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:30.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.770+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:30.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:30.770+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c006450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:31.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:31.171+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:31.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:31.171+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:31.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:31.171+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c006450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:31.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:31.972+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:31.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:31.972+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:31.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:31.972+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c006450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:33.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c006450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c006450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c00bfb0 2023-10-12T05:15:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.569+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:33.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.770+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:33.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.770+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:33.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:33.770+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:34.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:34.171+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:34.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:34.171+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:34.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:34.171+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:34.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:34.972+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:34.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:34.972+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:34.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:34.972+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:36.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:36.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:36.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:36.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:36.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c00bfb0 2023-10-12T05:15:36.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:36.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:36.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:36.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.570+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:36.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.771+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:36.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.771+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:36.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:36.771+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:37.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:37.172+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:37.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:37.172+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:37.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:37.172+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:37.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:37.973+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:37.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:37.973+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:37.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:37.973+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:39.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:39.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:39.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:39.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:39.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c00bfb0 2023-10-12T05:15:39.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:39.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:39.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:39.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.570+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:39.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.771+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:39.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.771+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:39.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:39.771+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:40.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:40.172+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:40.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:40.172+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:40.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:40.172+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:40.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:40.973+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:40.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:40.973+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:40.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:40.973+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:42.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:42.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:42.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00bfb0 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:42.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:42.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c006450 2023-10-12T05:15:42.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:42.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:42.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:42.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.569+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.770+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:42.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.770+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:42.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:42.770+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:43.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:43.171+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:43.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:43.171+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:43.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:43.171+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:43.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:43.972+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:43.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:43.972+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:43.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:43.972+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c006450 2023-10-12T05:15:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.570+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:45.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.771+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:45.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.771+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:45.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:45.771+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:46.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:46.172+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:46.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:46.172+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:46.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:46.172+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:46.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:46.973+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:46.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:46.973+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:46.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:46.973+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:48.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:48.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:48.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:48.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:48.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c006860 2023-10-12T05:15:48.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:48.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:48.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:48.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.570+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:48.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.771+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:48.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.771+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:48.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:48.771+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:49.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:49.172+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:49.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:49.172+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:49.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:49.172+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:49.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:49.973+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:49.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:49.973+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:49.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:49.973+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:51.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:51.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:51.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:51.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:51.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c006860 2023-10-12T05:15:51.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:51.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:51.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:51.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.570+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:51.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.771+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:51.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.771+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:51.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:51.771+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:52.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:52.172+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:52.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:52.172+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:52.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:52.172+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:52.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:52.973+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:52.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:52.973+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:52.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:52.973+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:54.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.570+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:54.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.570+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:54.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.570+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:54.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:54.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.571+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c006860 2023-10-12T05:15:54.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.571+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:54.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.571+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:54.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.571+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:54.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.571+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:54.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.772+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:54.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.772+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:54.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:54.772+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:55.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:55.173+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:55.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:55.173+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:55.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:55.173+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:55.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:55.974+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:55.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:55.974+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:55.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:55.974+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:15:57.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:57.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:15:57.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006860 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:15:57.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:15:57.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c006450 2023-10-12T05:15:57.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:15:57.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:57.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:57.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.571+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:15:57.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.772+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:57.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.772+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:57.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:57.772+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:15:58.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:58.173+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:58.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:58.173+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:58.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:58.173+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:15:58.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:58.974+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:15:58.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:58.974+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:15:58.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:15:58.974+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:00.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:00.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:00.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:00.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c005d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:00.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c006450 2023-10-12T05:16:00.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:00.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:00.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:00.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.571+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:00.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.772+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:00.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.772+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:00.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:00.772+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:01.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:01.173+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:01.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:01.173+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:01.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:01.173+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:01.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:01.974+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:01.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:01.974+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:01.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:01.974+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:03.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:03.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c005d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:03.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:03.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:03.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c006450 2023-10-12T05:16:03.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:03.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:03.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:03.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.571+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:03.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.772+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:03.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.772+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:03.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:03.772+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:04.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:04.173+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:04.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:04.173+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:04.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:04.173+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:04.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:04.974+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:04.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:04.974+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:04.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:04.974+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:06.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.571+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:06.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:06.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.571+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c009d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:06.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.571+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:06.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.572+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c006450 2023-10-12T05:16:06.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.572+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:06.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.572+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:06.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.572+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:06.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.572+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:06.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.773+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:06.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.773+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:06.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:06.773+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:07.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:07.174+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:07.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:07.174+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:07.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:07.174+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:07.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:07.975+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:07.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:07.975+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:07.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:07.975+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:09.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:09.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:09.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:09.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:09.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c006450 2023-10-12T05:16:09.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:09.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:09.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:09.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.572+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:09.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.773+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:09.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.773+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:09.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:09.773+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:10.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:10.174+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:10.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:10.174+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:10.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:10.174+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:10.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:10.975+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:10.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:10.975+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:10.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:10.975+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c006450 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c00ad60 2023-10-12T05:16:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.572+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:12.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.773+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:12.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.773+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:12.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:12.773+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:13.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:13.174+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:13.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:13.174+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:13.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:13.174+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:13.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:13.975+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:13.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:13.975+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:13.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:13.975+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:15.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:15.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:15.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:15.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:15.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c005d40 2023-10-12T05:16:15.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:15.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:15.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:15.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.573+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:15.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.774+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:15.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.774+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:15.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:15.774+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:16.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:16.175+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:16.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:16.175+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:16.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:16.175+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:16.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:16.976+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:16.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:16.976+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:16.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:16.976+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:18.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c005d40 2023-10-12T05:16:18.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:18.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:18.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:18.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.573+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:18.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.774+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:18.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.774+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:18.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:18.774+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:19.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:19.175+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:19.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:19.175+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:19.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:19.175+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:19.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:19.976+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:19.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:19.976+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:19.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:19.976+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:21.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:21.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:21.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:21.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:21.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c005d40 2023-10-12T05:16:21.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:16:21.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00e8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:21.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:21.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.573+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:21.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.774+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00e8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:21.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.774+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:21.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:21.774+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:22.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:22.175+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00e8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:22.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:22.175+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:22.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:22.175+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:22.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:22.976+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00e8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:22.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:22.976+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:22.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:22.976+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:24.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00e8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:24.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:24.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:24.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:24.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c005d40 2023-10-12T05:16:24.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:16:24.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:24.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:24.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.574+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:24.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.775+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:24.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.775+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:24.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:24.775+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:25.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:25.176+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:25.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:25.176+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:25.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:25.176+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:25.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:25.977+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:25.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:25.977+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:25.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:25.977+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:27.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:27.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c005d40 2023-10-12T05:16:27.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:16:27.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:27.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:27.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.574+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:27.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.775+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:27.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.775+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:27.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:27.775+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:28.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:28.176+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:28.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:28.176+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:28.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:28.176+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:28.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:28.977+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:28.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:28.977+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:28.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:28.977+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c005d40 2023-10-12T05:16:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c00ad60 2023-10-12T05:16:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.574+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:30.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.775+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:30.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.775+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:30.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:30.775+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:31.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:31.176+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:31.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:31.176+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:31.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:31.176+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:31.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:31.977+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:31.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:31.977+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:31.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:31.977+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:33.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:33.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c00ad60 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c010900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c005d40 2023-10-12T05:16:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.575+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c010900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:33.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.776+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:33.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.776+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:33.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:33.776+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c010900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:34.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:34.177+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:34.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:34.177+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:34.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:34.177+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c010900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:34.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:34.978+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:34.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:34.978+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:34.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:34.978+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c010900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:36.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c010900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:36.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c010900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:36.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:36.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:36.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c005d40 2023-10-12T05:16:36.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:36.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:36.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:36.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.574+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:36.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.775+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.775+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:36.775+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:37.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:37.176+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:37.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:37.176+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:37.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:37.176+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:37.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:37.977+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:37.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:37.977+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:37.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:37.977+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:39.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:39.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:39.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c005d40 legacy=0x7f392c006150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:39.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:39.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c00ad60 2023-10-12T05:16:39.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:39.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:39.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:39.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.574+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:39.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.775+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:39.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.775+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:39.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:39.775+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:40.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:40.176+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:40.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:40.176+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:40.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:40.176+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:40.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:40.977+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:40.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:40.977+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:40.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:40.977+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:42.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.574+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:42.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.574+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:42.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.574+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c010900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:42.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.575+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:42.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.575+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806e400 con 0x7f392c00ad60 2023-10-12T05:16:42.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.575+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:42.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.575+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:42.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.575+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:42.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.575+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:42.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.776+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:42.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.776+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:42.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:42.776+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:43.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:43.177+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:43.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:43.177+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:43.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:43.177+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:43.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:43.978+0000 7f394d1a1640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:43.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:43.978+0000 7f394d9a2640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:43.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:43.978+0000 7f394d9a2640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:45.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f393f7fe640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:45.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-12T05:16:45.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f393f7fe640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-12T05:16:45.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f393f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-12T05:16:45.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f393f7fe640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394806dcb0 con 0x7f392c00ad60 2023-10-12T05:16:45.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f393f7fe640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394809a630 con 0x7f392c006860 2023-10-12T05:16:45.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:45.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:45.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.575+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-12T05:16:45.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.776+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:45.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.776+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:45.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:45.776+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-12T05:16:46.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:46.177+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:46.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:46.177+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:46.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:46.177+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-12T05:16:46.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:46.978+0000 7f394d1a1640 1 -- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 msgr2=0x7f392c00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.125:3300/0 2023-10-12T05:16:46.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:46.978+0000 7f394d9a2640 1 -- >> v1:172.21.15.125:6789/0 conn(0x7f392c00ad60 legacy=0x7f392c005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.125:6789/0 2023-10-12T05:16:46.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-12T05:16:46.978+0000 7f394d1a1640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f392c006860 0x7f392c00ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-12T05:16:48.489 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-12T05:16:48.490 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-12T05:16:48.499 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-12T05:16:48.499 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-12T05:16:48.500 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-12T05:16:48.500 INFO:teuthology.orchestra.run:waiting for 300 2023-10-12T05:16:48.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:48.503 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log' 2023-10-12T05:16:48.511 INFO:tasks.ceph.osd.0:Stopped 2023-10-12T05:16:48.512 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-12T05:16:48.512 INFO:teuthology.orchestra.run:waiting for 300 2023-10-12T05:16:48.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:48.513 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log' 2023-10-12T05:16:48.521 INFO:tasks.ceph.osd.1:Stopped 2023-10-12T05:16:48.522 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-12T05:16:48.522 INFO:teuthology.orchestra.run:waiting for 300 2023-10-12T05:16:48.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:48.523 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi158 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log' 2023-10-12T05:16:48.532 INFO:tasks.ceph.osd.2:Stopped 2023-10-12T05:16:48.532 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-12T05:16:48.533 INFO:teuthology.orchestra.run:waiting for 300 2023-10-12T05:16:48.611 INFO:tasks.ceph.osd.3:Stopped 2023-10-12T05:16:48.611 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-12T05:16:48.612 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-12T05:16:48.612 INFO:teuthology.orchestra.run:waiting for 300 2023-10-12T05:16:48.613 INFO:tasks.ceph.mgr.x:Stopped 2023-10-12T05:16:48.613 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-12T05:16:48.614 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-12T05:16:48.614 INFO:teuthology.orchestra.run:waiting for 300 2023-10-12T05:16:48.615 INFO:tasks.ceph.mon.a:Stopped 2023-10-12T05:16:48.615 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-12T05:16:48.616 DEBUG:teuthology.orchestra.run.smithi125:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2023-10-12T05:16:48.662 INFO:teuthology.orchestra.run.smithi125.stdout:2023-10-12T02:15:18.896817+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2023-10-12T05:16:48.663 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-12T05:16:48.663 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-12T05:16:48.664 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-12T05:16:48.753 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-12T05:16:48.754 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-12T05:16:48.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi158.front.sepia.ceph.com 2023-10-12T05:16:48.828 DEBUG:teuthology.orchestra.run.smithi158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-12T05:16:48.909 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi158.front.sepia.ceph.com 2023-10-12T05:16:48.910 DEBUG:teuthology.orchestra.run.smithi158:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-12T05:16:48.974 INFO:tasks.ceph:Archiving mon data... 2023-10-12T05:16:48.976 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/data/mon.a.tgz 2023-10-12T05:16:48.976 DEBUG:teuthology.orchestra.run.smithi125:> mktemp 2023-10-12T05:16:48.993 INFO:teuthology.orchestra.run.smithi125.stdout:/tmp/tmp.TtEzRTd8Vi 2023-10-12T05:16:48.993 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.TtEzRTd8Vi 2023-10-12T05:16:49.231 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0666 /tmp/tmp.TtEzRTd8Vi 2023-10-12T05:16:49.316 DEBUG:teuthology.orchestra.remote:smithi125:/tmp/tmp.TtEzRTd8Vi is 807KB 2023-10-12T05:16:49.418 DEBUG:teuthology.orchestra.run.smithi125:> rm -fr /tmp/tmp.TtEzRTd8Vi 2023-10-12T05:16:49.432 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-12T05:16:49.433 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-12T05:16:49.474 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-12T05:16:49.507 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-12T05:16:49.508 DEBUG:teuthology.orchestra.run.smithi125:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-12T05:16:49.541 DEBUG:teuthology.orchestra.run.smithi158:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-12T05:16:49.566 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-12T05:16:49.580 INFO:teuthology.orchestra.run.smithi158.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-12T05:16:49.584 INFO:tasks.ceph:Archiving crash dumps... 2023-10-12T05:16:49.586 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/crash to /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/remote/smithi125/crash 2023-10-12T05:16:49.588 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-12T05:16:49.637 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/lib/ceph/crash to /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/remote/smithi158/crash 2023-10-12T05:16:49.638 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-12T05:16:49.681 INFO:tasks.ceph:Compressing logs... 2023-10-12T05:16:49.681 DEBUG:teuthology.orchestra.run.smithi125:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-12T05:16:49.684 DEBUG:teuthology.orchestra.run.smithi158:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-12T05:16:49.708 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-12T05:16:49.708 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82834.log 2023-10-12T05:16:49.709 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83648.log 2023-10-12T05:16:49.709 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81828.log 2023-10-12T05:16:49.709 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.79245.log 2023-10-12T05:16:49.712 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84346.log 2023-10-12T05:16:49.713 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph.tmp-client.admin.79245.log: gzip -5 --verbose -- 86.3% /var/log/ceph/ceph-client.admin.83733.log 2023-10-12T05:16:49.713 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82834.log.gz 2023-10-12T05:16:49.713 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83648.log: /var/log/ceph/ceph-client.admin.81828.log: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.79245.log.gz 2023-10-12T05:16:49.713 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84493.log 2023-10-12T05:16:49.714 INFO:teuthology.orchestra.run.smithi125.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.81828.log.gz 2023-10-12T05:16:49.714 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83699.log 2023-10-12T05:16:49.714 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84310.log 2023-10-12T05:16:49.715 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83850.log 2023-10-12T05:16:49.715 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83737.log 2023-10-12T05:16:49.715 INFO:teuthology.orchestra.run.smithi125.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.83699.log.gz/var/log/ceph/ceph-client.admin.83850.log: 2023-10-12T05:16:49.715 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83956.log 2023-10-12T05:16:49.717 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83737.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.84310.log.gz 2023-10-12T05:16:49.717 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84435.log 2023-10-12T05:16:49.727 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84373.log 2023-10-12T05:16:49.731 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84435.log: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.84493.log.gz 2023-10-12T05:16:49.733 INFO:teuthology.orchestra.run.smithi125.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.83737.log.gz 2023-10-12T05:16:49.734 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84470.log 2023-10-12T05:16:49.735 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84373.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.84373.log.gz 2023-10-12T05:16:49.735 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-12T05:16:49.737 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84470.log: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.83956.log.gz 2023-10-12T05:16:49.738 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84098.log 2023-10-12T05:16:49.740 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph.audit.log: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.83648.log.gz 2023-10-12T05:16:49.740 INFO:teuthology.orchestra.run.smithi125.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84469.log 2023-10-12T05:16:49.741 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-12T05:16:49.741 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84188.log 2023-10-12T05:16:49.743 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84469.log: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.83733.log.gz 2023-10-12T05:16:49.744 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83678.log 2023-10-12T05:16:49.748 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84188.log: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.84346.log.gz 2023-10-12T05:16:49.749 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83963.log 2023-10-12T05:16:49.750 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83678.log: 96.5% -- replaced with /var/log/ceph/ceph-client.admin.83850.log.gz 2023-10-12T05:16:49.751 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84480.log 2023-10-12T05:16:49.753 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81587.log 2023-10-12T05:16:49.753 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81627.log 2023-10-12T05:16:49.754 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-12T05:16:49.754 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/ceph-client.admin.81587.log: /var/log/ceph/ceph-client.admin.81627.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-12T05:16:49.754 INFO:teuthology.orchestra.run.smithi158.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.81587.log.gz 2023-10-12T05:16:49.755 INFO:teuthology.orchestra.run.smithi158.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.81627.log.gz 2023-10-12T05:16:49.755 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81667.log 2023-10-12T05:16:49.755 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-12T05:16:49.755 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/ceph-client.admin.81667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81547.log 2023-10-12T05:16:49.756 INFO:teuthology.orchestra.run.smithi158.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.81667.log.gz 2023-10-12T05:16:49.757 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.81547.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.81547.log.gz 2023-10-12T05:16:49.758 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83963.log: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.84470.log.gz 2023-10-12T05:16:49.758 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81565.log 2023-10-12T05:16:49.764 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84480.log: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.84435.log.gz 2023-10-12T05:16:49.764 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83146.log 2023-10-12T05:16:49.765 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.81565.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.81565.log.gz 2023-10-12T05:16:49.766 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82876.log 2023-10-12T05:16:49.767 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83146.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.83146.log.gz 2023-10-12T05:16:49.767 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84039.log 2023-10-12T05:16:49.767 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82876.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.82876.log.gz 2023-10-12T05:16:49.768 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-12T05:16:49.774 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84039.log: 96.5% -- replaced with /var/log/ceph/ceph-client.admin.84469.log.gz 2023-10-12T05:16:49.775 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82977.log 2023-10-12T05:16:49.780 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-osd.1.log: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.84188.log.gz 2023-10-12T05:16:49.780 INFO:teuthology.orchestra.run.smithi125.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.84098.log.gz 2023-10-12T05:16:49.781 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83721.log 2023-10-12T05:16:49.781 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82792.log 2023-10-12T05:16:49.781 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83721.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.82977.log.gz 2023-10-12T05:16:49.782 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82708.log 2023-10-12T05:16:49.782 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82792.log: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.83963.log.gz 2023-10-12T05:16:49.782 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.82792.log.gz 2023-10-12T05:16:49.791 INFO:teuthology.orchestra.run.smithi125.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.84480.log.gz 2023-10-12T05:16:49.791 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81644.log 2023-10-12T05:16:49.792 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83296.log 2023-10-12T05:16:49.792 INFO:teuthology.orchestra.run.smithi125.stderr: 86.3%/var/log/ceph/ceph-client.admin.81644.log: -- replaced with /var/log/ceph/ceph-client.admin.82708.log.gz 2023-10-12T05:16:49.792 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84146.log 2023-10-12T05:16:49.792 INFO:teuthology.orchestra.run.smithi125.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.81644.log.gz 2023-10-12T05:16:49.793 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82750.log 2023-10-12T05:16:49.793 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85903.log 2023-10-12T05:16:49.793 INFO:teuthology.orchestra.run.smithi125.stderr: 85.7%/var/log/ceph/ceph-client.admin.84146.log: -- replaced with /var/log/ceph/ceph-client.admin.83296.log.gz 2023-10-12T05:16:49.793 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82976.log 2023-10-12T05:16:49.794 INFO:teuthology.orchestra.run.smithi125.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.82750.log.gz 2023-10-12T05:16:49.794 INFO:teuthology.orchestra.run.smithi125.stderr: 96.4%/var/log/ceph/ceph-client.admin.85903.log: -- replaced with /var/log/ceph/ceph-client.admin.83678.log.gz 2023-10-12T05:16:49.794 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83338.log 2023-10-12T05:16:49.795 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83143.log 2023-10-12T05:16:49.795 INFO:teuthology.orchestra.run.smithi125.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.84146.log.gz 2023-10-12T05:16:49.795 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82976.log.gz 2023-10-12T05:16:49.795 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81800.log 2023-10-12T05:16:49.796 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.83338.log.gz 2023-10-12T05:16:49.796 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83143.log: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.85903.log.gz 2023-10-12T05:16:49.796 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84107.log 2023-10-12T05:16:49.797 INFO:teuthology.orchestra.run.smithi125.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.83143.log.gz 2023-10-12T05:16:49.797 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.81800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83254.log 2023-10-12T05:16:49.797 INFO:teuthology.orchestra.run.smithi125.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.81800.log.gz 2023-10-12T05:16:49.797 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83899.log 2023-10-12T05:16:49.798 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83677.log 2023-10-12T05:16:49.798 INFO:teuthology.orchestra.run.smithi125.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.83254.log.gz 2023-10-12T05:16:49.798 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84142.log 2023-10-12T05:16:49.799 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83630.log 2023-10-12T05:16:49.799 INFO:teuthology.orchestra.run.smithi125.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83899.log.gz 2023-10-12T05:16:49.799 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83677.log: /var/log/ceph/ceph-client.admin.84142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83823.log 2023-10-12T05:16:49.803 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83630.log: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.84039.log.gz 2023-10-12T05:16:49.803 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-12T05:16:49.804 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83823.log: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.83630.log.gz 2023-10-12T05:16:49.804 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82975.log 2023-10-12T05:16:49.804 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-osd.0.log: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.83721.log.gz 2023-10-12T05:16:49.805 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83981.log 2023-10-12T05:16:49.805 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82975.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82975.log.gz 2023-10-12T05:16:49.805 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-12T05:16:49.806 INFO:teuthology.orchestra.run.smithi158.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-12T05:16:49.810 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83981.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.83981.log.gz 2023-10-12T05:16:49.810 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82978.log 2023-10-12T05:16:49.810 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-12T05:16:49.811 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83145.log 2023-10-12T05:16:49.811 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82978.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.82978.log.gz 2023-10-12T05:16:49.811 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84492.log 2023-10-12T05:16:49.812 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83145.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.83145.log.gz 2023-10-12T05:16:49.812 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84481.log 2023-10-12T05:16:49.818 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84492.log: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.83823.log.gz 2023-10-12T05:16:49.819 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82644.log 2023-10-12T05:16:49.819 INFO:teuthology.orchestra.run.smithi125.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.admin.84107.log.gz 2023-10-12T05:16:49.820 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84277.log 2023-10-12T05:16:49.821 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.82644.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.82644.log.gz 2023-10-12T05:16:49.821 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83144.log 2023-10-12T05:16:49.822 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.84277.log: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.84142.log.gz 2023-10-12T05:16:49.823 INFO:teuthology.orchestra.run.smithi125.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.84277.log.gz 2023-10-12T05:16:49.823 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.83144.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.83144.log.gz 2023-10-12T05:16:49.828 INFO:teuthology.orchestra.run.smithi125.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.83677.log.gz 2023-10-12T05:16:49.834 INFO:teuthology.orchestra.run.smithi125.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.84481.log.gz 2023-10-12T05:16:49.835 INFO:teuthology.orchestra.run.smithi125.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.admin.84492.log.gz 2023-10-12T05:16:49.865 INFO:teuthology.orchestra.run.smithi158.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-12T05:16:50.054 INFO:teuthology.orchestra.run.smithi125.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-12T05:16:50.097 INFO:teuthology.orchestra.run.smithi125.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-12T05:16:51.022 INFO:teuthology.orchestra.run.smithi125.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-12T05:16:51.024 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-12T05:16:51.024 INFO:teuthology.orchestra.run.smithi125.stderr:real 0m1.328s 2023-10-12T05:16:51.024 INFO:teuthology.orchestra.run.smithi125.stderr:user 0m2.490s 2023-10-12T05:16:51.025 INFO:teuthology.orchestra.run.smithi125.stderr:sys 0m0.164s 2023-10-12T05:16:51.476 INFO:teuthology.orchestra.run.smithi158.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-12T05:16:51.479 INFO:teuthology.orchestra.run.smithi158.stderr: 2023-10-12T05:16:51.479 INFO:teuthology.orchestra.run.smithi158.stderr:real 0m1.740s 2023-10-12T05:16:51.480 INFO:teuthology.orchestra.run.smithi158.stderr:user 0m1.815s 2023-10-12T05:16:51.480 INFO:teuthology.orchestra.run.smithi158.stderr:sys 0m0.105s 2023-10-12T05:16:51.480 INFO:tasks.ceph:Archiving logs... 2023-10-12T05:16:51.481 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/log/ceph to /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/remote/smithi125/log 2023-10-12T05:16:51.483 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-12T05:16:51.878 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/log/ceph to /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/remote/smithi158/log 2023-10-12T05:16:51.880 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-12T05:16:52.187 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-12T05:16:52.196 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-12T05:16:52.204 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-12T05:16:52.209 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-12T05:16:52.209 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-12T05:16:52.212 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-12T05:16:52.291 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-12T05:16:52.299 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-12T05:16:52.300 DEBUG:teuthology.orchestra.run.smithi125:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-12T05:16:52.303 DEBUG:teuthology.orchestra.run.smithi158:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-12T05:16:52.314 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2023-10-12T05:16:52.317 INFO:teuthology.orchestra.run.smithi158.stderr:bash: line 1: ntpq: command not found 2023-10-12T05:16:52.318 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-12T05:16:52.319 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2023-10-12T05:16:52.319 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv01.front.sepia.ceph.com 2 10 377 37 -1593us[-1593us] +/- 89ms 2023-10-12T05:16:52.319 INFO:teuthology.orchestra.run.smithi125.stdout:^* hv02.front.sepia.ceph.com 3 7 377 45 +239us[ +232us] +/- 83ms 2023-10-12T05:16:52.320 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 48 +1355us[+1348us] +/- 85ms 2023-10-12T05:16:52.320 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T05:16:52.322 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-12T05:16:52.322 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2023-10-12T05:16:52.323 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv01.front.sepia.ceph.com 2 9 377 378 -1617us[-1592us] +/- 83ms 2023-10-12T05:16:52.323 INFO:teuthology.orchestra.run.smithi158.stdout:^* hv02.front.sepia.ceph.com 3 8 377 166 +190us[ +214us] +/- 81ms 2023-10-12T05:16:52.323 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 189 +1278us[+1302us] +/- 82ms 2023-10-12T05:16:52.323 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-12T05:16:52.324 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-12T05:16:52.332 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-12T05:16:52.333 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-12T05:16:52.346 DEBUG:teuthology.orchestra.run.smithi125:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-12T05:16:52.453 DEBUG:teuthology.orchestra.run.smithi158:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-12T05:16:52.542 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-12T05:16:52.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:52.574 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-12T05:16:52.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:52.605 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-12T05:16:52.614 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-12T05:16:52.769 INFO:teuthology.task.internal:Duration was 11593.975803 seconds 2023-10-12T05:16:52.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-12T05:16:52.781 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-12T05:16:52.782 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-12T05:16:52.785 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-12T05:16:52.826 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-12T05:16:52.828 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-12T05:16:53.288 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-12T05:16:53.288 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi125.front.sepia.ceph.com 2023-10-12T05:16:53.289 DEBUG:teuthology.orchestra.run.smithi125:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-12T05:16:53.309 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi158.front.sepia.ceph.com 2023-10-12T05:16:53.309 DEBUG:teuthology.orchestra.run.smithi158:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-12T05:16:53.355 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-12T05:16:53.355 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-12T05:16:53.358 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-12T05:16:53.424 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-12T05:16:53.424 DEBUG:teuthology.orchestra.run.smithi125:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-12T05:16:53.427 DEBUG:teuthology.orchestra.run.smithi158:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-12T05:16:53.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-12T05:16:53.571 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-12T05:16:53.572 DEBUG:teuthology.orchestra.run.smithi125:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-12T05:16:53.599 DEBUG:teuthology.orchestra.run.smithi158:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-12T05:16:53.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-12T05:16:53.637 DEBUG:teuthology.orchestra.run.smithi125:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-12T05:16:53.641 DEBUG:teuthology.orchestra.run.smithi158:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-12T05:16:53.664 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = core 2023-10-12T05:16:53.691 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = core 2023-10-12T05:16:53.734 DEBUG:teuthology.orchestra.run.smithi125:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-12T05:16:53.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:53.749 DEBUG:teuthology.orchestra.run.smithi158:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-12T05:16:53.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:53.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-12T05:16:53.800 INFO:teuthology.task.internal:Transferring archived files... 2023-10-12T05:16:53.801 DEBUG:teuthology.misc:Transferring archived files from smithi125:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/remote/smithi125 2023-10-12T05:16:53.801 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-12T05:16:53.837 DEBUG:teuthology.misc:Transferring archived files from smithi158:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367/remote/smithi158 2023-10-12T05:16:53.838 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-12T05:16:53.873 INFO:teuthology.task.internal:Removing archive directory... 2023-10-12T05:16:53.874 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-12T05:16:53.876 DEBUG:teuthology.orchestra.run.smithi158:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-12T05:16:53.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-12T05:16:53.933 INFO:teuthology.task.internal:Not uploading archives. 2023-10-12T05:16:53.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-12T05:16:53.945 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-12T05:16:53.946 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-12T05:16:53.948 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-12T05:16:53.960 INFO:teuthology.orchestra.run.smithi125.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 12 05:16 /home/ubuntu/cephtest 2023-10-12T05:16:53.961 INFO:teuthology.orchestra.run.smithi125.stdout: 533596 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 12 02:13 /home/ubuntu/cephtest/mnt.0 2023-10-12T05:16:53.961 INFO:teuthology.orchestra.run.smithi125.stdout: 533597 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0 2023-10-12T05:16:53.961 INFO:teuthology.orchestra.run.smithi125.stdout: 545488 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-12T05:16:53.962 INFO:teuthology.orchestra.run.smithi125.stdout: 545693 60 -rw-r--r-- 1 ubuntu ubuntu 59016 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_tier_pp.log 2023-10-12T05:16:53.962 INFO:teuthology.orchestra.run.smithi125.stdout: 546920 4 -rw-r--r-- 1 ubuntu ubuntu 570 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_stat_pp.log 2023-10-12T05:16:53.962 INFO:teuthology.orchestra.run.smithi125.stdout: 545550 4 -rw-r--r-- 1 ubuntu ubuntu 208 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_lock.log 2023-10-12T05:16:53.962 INFO:teuthology.orchestra.run.smithi125.stdout: 547508 4 -rw-r--r-- 1 ubuntu ubuntu 1234 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_c_read_operations.log 2023-10-12T05:16:53.963 INFO:teuthology.orchestra.run.smithi125.stdout: 547507 0 -rw-r--r-- 1 ubuntu ubuntu 0 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_list_parallel.log 2023-10-12T05:16:53.963 INFO:teuthology.orchestra.run.smithi125.stdout: 545673 4 -rw-r--r-- 1 ubuntu ubuntu 1034 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_misc_pp.log 2023-10-12T05:16:53.963 INFO:teuthology.orchestra.run.smithi125.stdout: 545499 4 -rw-r--r-- 1 ubuntu ubuntu 696 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_aio.log 2023-10-12T05:16:53.964 INFO:teuthology.orchestra.run.smithi125.stdout: 545554 4 -rw-r--r-- 1 ubuntu ubuntu 3279 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_misc.log 2023-10-12T05:16:53.964 INFO:teuthology.orchestra.run.smithi125.stdout: 547506 4 -rw-r--r-- 1 ubuntu ubuntu 1171 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_cls_remote_reads.log 2023-10-12T05:16:53.964 INFO:teuthology.orchestra.run.smithi125.stdout: 547502 4 -rw-r--r-- 1 ubuntu ubuntu 361 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_cmd_pp.log 2023-10-12T05:16:53.964 INFO:teuthology.orchestra.run.smithi125.stdout: 547500 4 -rw-r--r-- 1 ubuntu ubuntu 351 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_cmd.log 2023-10-12T05:16:53.965 INFO:teuthology.orchestra.run.smithi125.stdout: 547463 4 -rw-r--r-- 1 ubuntu ubuntu 1121 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_watch_notify.log 2023-10-12T05:16:53.965 INFO:teuthology.orchestra.run.smithi125.stdout: 546796 4 -rw-r--r-- 1 ubuntu ubuntu 397 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_snapshots.log 2023-10-12T05:16:53.965 INFO:teuthology.orchestra.run.smithi125.stdout: 545549 28 -rw-r--r-- 1 ubuntu ubuntu 26305 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_asio.log 2023-10-12T05:16:53.965 INFO:teuthology.orchestra.run.smithi125.stdout: 547499 4 -rw-r--r-- 1 ubuntu ubuntu 345 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_stat.log 2023-10-12T05:16:53.966 INFO:teuthology.orchestra.run.smithi125.stdout: 547509 0 -rw-r--r-- 1 ubuntu ubuntu 0 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_delete_pools_parallel.log 2023-10-12T05:16:53.966 INFO:teuthology.orchestra.run.smithi125.stdout: 547503 4 -rw-r--r-- 1 ubuntu ubuntu 1087 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_service_pp.log 2023-10-12T05:16:53.966 INFO:teuthology.orchestra.run.smithi125.stdout: 547505 4 -rw-r--r-- 1 ubuntu ubuntu 956 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_c_write_operations.log 2023-10-12T05:16:53.966 INFO:teuthology.orchestra.run.smithi125.stdout: 547504 4 -rw-r--r-- 1 ubuntu ubuntu 2364 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_service.log 2023-10-12T05:16:53.967 INFO:teuthology.orchestra.run.smithi125.stdout: 545513 4 -rw-r--r-- 1 ubuntu ubuntu 283 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_aio_pp.log 2023-10-12T05:16:53.967 INFO:teuthology.orchestra.run.smithi125.stdout: 545529 4 -rw-r--r-- 1 ubuntu ubuntu 638 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_io_pp.log 2023-10-12T05:16:53.967 INFO:teuthology.orchestra.run.smithi125.stdout: 547510 0 -rw-r--r-- 1 ubuntu ubuntu 0 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_open_pools_parallel.log 2023-10-12T05:16:53.968 INFO:teuthology.orchestra.run.smithi125.stdout: 547501 8 -rw-r--r-- 1 ubuntu ubuntu 6062 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_watch_notify_pp.log 2023-10-12T05:16:53.968 INFO:teuthology.orchestra.run.smithi125.stdout: 545551 4 -rw-r--r-- 1 ubuntu ubuntu 225 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_lock_pp.log 2023-10-12T05:16:53.968 INFO:teuthology.orchestra.run.smithi125.stdout: 546862 4 -rw-r--r-- 1 ubuntu ubuntu 1611 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_pool.log 2023-10-12T05:16:53.968 INFO:teuthology.orchestra.run.smithi125.stdout: 545552 4 -rw-r--r-- 1 ubuntu ubuntu 547 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_list.log 2023-10-12T05:16:53.969 INFO:teuthology.orchestra.run.smithi125.stdout: 545922 4 -rw-r--r-- 1 ubuntu ubuntu 330 Oct 12 02:14 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_snapshots_pp.log 2023-10-12T05:16:53.969 INFO:teuthology.orchestra.run.smithi125.stdout: 545547 4 -rw-r--r-- 1 ubuntu ubuntu 1270 Oct 12 02:15 /home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph_test_rados_api_io.log 2023-10-12T05:16:53.970 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-12T05:16:53.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-12T05:16:53.971 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-12T05:16:53.982 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-12T05:16:53.994 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-12T05:16:54.008 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-10-12T05:16:54.008 INFO:teuthology.nuke:Checking targets against current locks 2023-10-12T05:16:54.013 INFO:teuthology.orchestra.run.smithi158.stdout: 265982 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 12 05:16 /home/ubuntu/cephtest 2023-10-12T05:16:54.036 DEBUG:teuthology.nuke:shortname: smithi125 2023-10-12T05:16:54.036 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-12T05:16:54.060 DEBUG:teuthology.nuke:shortname: smithi158 2023-10-12T05:16:54.061 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-12T05:16:54.064 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-12 01:54:32.373950', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN73DjrdE1g/zWHtmfxoMBCa2KEZsi38QXldujS7u0Zww7nZEB2UwjO8NtmyYYBxfjV9JEweM2TVFCIb8a4Mc+8='} 2023-10-12T05:16:54.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2023-10-12_01:35:43-crimson-rados-wip-yingxin-crimson-osd-crosscore-pg-submission-distro-default-smithi/7422367', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-12 01:54:32.375532', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCBwCQt/Ud2a8XaZeiwNWB9DezxlH0sYlAKXUvpN15HLpscSYamtBdB1EmEAaFCHN7visu6WEigz8M/5Vw1NYI='} 2023-10-12T05:16:54.118 INFO:teuthology.orchestra.console.smithi125:Power off 2023-10-12T05:16:54.119 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-12T05:16:54.141 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2023-10-12T05:16:54.141 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T05:16:54.155 INFO:teuthology.orchestra.console.smithi158:Power off 2023-10-12T05:16:54.155 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-12T05:16:54.163 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-12T05:16:54.176 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2023-10-12T05:16:54.177 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T05:16:54.198 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-10-12T05:16:58.164 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T05:16:58.201 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T05:16:58.275 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-12T05:16:58.311 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-10-12T05:17:02.278 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T05:17:02.312 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T05:17:02.389 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-12T05:17:02.422 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-10-12T05:17:02.523 INFO:teuthology.orchestra.console.smithi158:Power off completed 2023-10-12T05:17:06.393 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-12T05:17:06.503 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-12T05:17:06.604 INFO:teuthology.orchestra.console.smithi125:Power off completed 2023-10-12T05:17:06.710 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 11593.975803375244 failure_reason: 'Command failed (workunit test rados/test.sh) on smithi125 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9587ffdfc8c52bcf06abbe89d2f527c434a3e9f5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh''' flavor: crimson owner: scheduled_yingxin@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7b33718969f94f738e77ad1dfecfdf5f status: fail success: false 2023-10-12T05:17:06.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-12T05:17:06.790 INFO:teuthology.run:FAIL