2023-05-23T07:58:40.145 INFO:root:teuthology version: 0.0.1.dev105+g6fb4f0c
2023-05-23T07:58:40.302 INFO:teuthology.lock.ops:Start node 'smithi042.front.sepia.ceph.com' reimaging
2023-05-23T07:58:40.303 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: reset os type and version on server
2023-05-23T07:58:40.303 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2023-05-23T07:58:40.328 INFO:teuthology.lock.ops:Node 'smithi042.front.sepia.ceph.com' reimaging is complete
2023-05-23T07:58:40.329 INFO:teuthology.lock.ops:Start node 'smithi105.front.sepia.ceph.com' reimaging
2023-05-23T07:58:40.329 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: reset os type and version on server
2023-05-23T07:58:40.329 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2023-05-23T07:58:40.353 INFO:teuthology.lock.ops:Node 'smithi105.front.sepia.ceph.com' reimaging is complete
2023-05-23T07:58:40.504 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of rhel 8.6
2023-05-23T07:58:40.508 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of rhel 8.6
2023-05-23T07:58:40.768 INFO:teuthology.orchestra.console:Power off smithi105
2023-05-23T07:58:40.812 INFO:teuthology.orchestra.console:Power off smithi042
2023-05-23T07:58:49.151 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2023-05-23T07:58:49.253 INFO:teuthology.orchestra.console:Power on smithi105
2023-05-23T07:58:53.293 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2023-05-23T07:58:53.395 INFO:teuthology.orchestra.console:Power on smithi042
2023-05-23T07:58:57.645 INFO:teuthology.orchestra.console:Power on for smithi105 completed
2023-05-23T07:58:57.646 INFO:teuthology.orchestra.console:Power on for smithi042 completed
2023-05-23T07:58:57.746 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish
2023-05-23T07:58:57.749 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish
2023-05-23T08:01:31.728 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.42
2023-05-23T08:01:40.784 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.42
2023-05-23T08:01:46.800 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.105
2023-05-23T08:01:49.840 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.42
2023-05-23T08:01:55.856 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.105
2023-05-23T08:01:58.896 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.42
2023-05-23T08:02:04.912 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.105
2023-05-23T08:02:07.952 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.42
2023-05-23T08:02:13.972 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.105
2023-05-23T08:02:17.008 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.42
2023-05-23T08:02:23.056 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.105
2023-05-23T08:02:32.112 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.105
2023-05-23T08:03:23.031 WARNING:teuthology.provision.fog:timed out
2023-05-23T08:03:29.371 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-23T08:03:29.698 INFO:teuthology.provision.fog.smithi042:Node is ready
2023-05-23T08:03:29.716 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com
2023-05-23T08:03:29.775 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042
2023-05-23T08:03:30.138 INFO:teuthology.provision.fog.smithi042:Deploy complete!
2023-05-23T08:03:38.123 WARNING:teuthology.provision.fog:timed out
2023-05-23T08:03:44.458 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-23T08:03:44.755 INFO:teuthology.provision.fog.smithi105:Node is ready
2023-05-23T08:03:44.772 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105
2023-05-23T08:03:44.829 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105
2023-05-23T08:03:45.176 INFO:teuthology.provision.fog.smithi105:Deploy complete!
2023-05-23T08:03:45.362 INFO:teuthology.lock.ops:Checking smithi042.front.sepia.ceph.com
2023-05-23T08:03:45.363 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-23T08:03:45.458 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com
2023-05-23T08:03:45.458 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-23T08:03:45.596 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: set os type and version on server
2023-05-23T08:03:45.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2023-05-23T08:03:45.857 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64
2023-05-23T08:03:45.914 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Red Hat Enterprise Linux"
2023-05-23T08:03:45.914 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8.6 (Ootpa)"
2023-05-23T08:03:45.915 INFO:teuthology.orchestra.run.smithi042.stdout:ID="rhel"
2023-05-23T08:03:45.915 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="fedora"
2023-05-23T08:03:45.915 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8.6"
2023-05-23T08:03:45.915 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8"
2023-05-23T08:03:45.915 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-05-23T08:03:45.915 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31"
2023-05-23T08:03:45.915 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.redhat.com/"
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-05-23T08:03:45.916 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-05-23T08:03:45.917 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2023-05-23T08:03:45.945 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server
2023-05-23T08:03:45.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2023-05-23T08:03:46.134 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64
2023-05-23T08:03:46.191 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux"
2023-05-23T08:03:46.192 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.6 (Ootpa)"
2023-05-23T08:03:46.192 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel"
2023-05-23T08:03:46.192 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora"
2023-05-23T08:03:46.192 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.6"
2023-05-23T08:03:46.193 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8"
2023-05-23T08:03:46.193 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-05-23T08:03:46.193 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31"
2023-05-23T08:03:46.193 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-05-23T08:03:46.194 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/"
2023-05-23T08:03:46.194 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-05-23T08:03:46.194 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-23T08:03:46.195 INFO:teuthology.orchestra.run.smithi105.stdout:
2023-05-23T08:03:46.195 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-23T08:03:46.196 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-05-23T08:03:46.196 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-05-23T08:03:46.196 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-05-23T08:03:46.197 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2023-05-23T08:03:46.335 INFO:teuthology.dispatcher.supervisor:Running job 7283474
2023-05-23T08:03:46.335 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6fb4f0c910de3902d27909a642e8d50d731dd28c/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-05-23_07:50:52-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7283474 --name mchangir-2023-05-23_07:50:52-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi --description fs:mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} -- /home/teuthworker/archive/mchangir-2023-05-23_07:50:52-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7283474/orig.config.yaml
2023-05-23T08:03:46.338 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2023-05-23_07:50:52-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7283474
2023-05-23T08:03:46.339 INFO:teuthology.dispatcher.supervisor:Job PID: 969
2023-05-23T08:03:46.339 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-05-23T20:05:11.406 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-05-23T20:05:11.550 INFO:teuthology.kill:Killing Pids: {969}
2023-05-23T20:05:12.423 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2023-05-23T20:05:12.424 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']
2023-05-23T20:05:12.424 INFO:teuthology.misc:Compressing logs...
2023-05-23T20:05:12.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2023-05-23T20:05:12.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2023-05-23T20:05:13.137 INFO:teuthology.misc:Compressing logs...
2023-05-23T20:05:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2023-05-23T20:05:13.210 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/home/ubuntu/cephtest/archive’: No such file or directory
2023-05-23T20:05:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2023-05-23T20:05:13.281 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now
2023-05-23T20:05:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /home/ubuntu/cephtest/archive: Cannot open: No such file or directory
2023-05-23T20:05:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now
2023-05-23T20:05:13.349 INFO:teuthology.misc:Compressing logs...
2023-05-23T20:05:21.510 INFO:teuthology.kill:No teuthology processes running
2023-05-23T20:05:22.313 INFO:teuthology.kill:Nuking machines: ['smithi042', 'smithi105']
2023-05-23T20:05:22.955 INFO:teuthology.kill:2023-05-23 20:05:22,954.954 INFO:teuthology.nuke:targets:
2023-05-23T20:05:22.955 INFO:teuthology.kill:  smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=
2023-05-23T20:05:22.956 INFO:teuthology.kill:  smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-05-23T20:05:22.956 INFO:teuthology.kill:2023-05-23 20:05:22,954.954 INFO:teuthology.nuke:Checking targets against current locks
2023-05-23T20:05:23.013 INFO:teuthology.kill:2023-05-23 20:05:23,013.013 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-05-23T20:05:23.030 INFO:teuthology.kill:2023-05-23 20:05:23,029.029 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-05-23T20:05:23.087 INFO:teuthology.kill:2023-05-23 20:05:23,087.087 INFO:teuthology.orchestra.console:Power off smithi042
2023-05-23T20:05:23.124 INFO:teuthology.kill:2023-05-23 20:05:23,123.123 INFO:teuthology.orchestra.console:Power off smithi105
2023-05-23T20:05:31.457 INFO:teuthology.kill:2023-05-23 20:05:31,456.456 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2023-05-23T20:05:31.611 INFO:teuthology.kill:2023-05-23 20:05:31,611.611 INFO:teuthology.lock.ops:unlocked: smithi042.front.sepia.ceph.com
2023-05-23T20:05:35.596 INFO:teuthology.kill:2023-05-23 20:05:35,596.596 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2023-05-23T20:05:35.752 INFO:teuthology.kill:2023-05-23 20:05:35,752.752 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com
2023-05-23T20:07:36.104 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-05-23T20:07:36.207 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi042 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-05-23_14:47:16-rbd-wip-yuri4-testing-2023-05-22-1631-quincy-distro-default-smithi/7283844
2023-05-23T20:07:36.207 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi105 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-05-23_14:47:16-rbd-wip-yuri4-testing-2023-05-22-1631-quincy-distro-default-smithi/7283845