2023-01-13T10:12:44.790 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2023-01-13T10:12:44.909 INFO:teuthology.lock.ops:Start node 'smithi043.front.sepia.ceph.com' reimaging 2023-01-13T10:12:44.910 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: reset os type and version on server 2023-01-13T10:12:44.910 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2023-01-13T10:12:44.925 INFO:teuthology.lock.ops:Node 'smithi043.front.sepia.ceph.com' reimaging is complete 2023-01-13T10:12:44.925 INFO:teuthology.lock.ops:Start node 'smithi106.front.sepia.ceph.com' reimaging 2023-01-13T10:12:44.926 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: reset os type and version on server 2023-01-13T10:12:44.926 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2023-01-13T10:12:44.941 INFO:teuthology.lock.ops:Node 'smithi106.front.sepia.ceph.com' reimaging is complete 2023-01-13T10:12:45.094 INFO:teuthology.provision.fog.smithi043:Scheduling deploy of rhel 8.6 2023-01-13T10:12:45.106 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of rhel 8.6 2023-01-13T10:12:45.378 INFO:teuthology.orchestra.console:Power off smithi043 2023-01-13T10:12:45.399 INFO:teuthology.orchestra.console:Power off smithi106 2023-01-13T10:12:53.765 INFO:teuthology.orchestra.console:Power off for smithi043 completed 2023-01-13T10:12:53.776 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2023-01-13T10:12:53.867 INFO:teuthology.orchestra.console:Power on smithi043 2023-01-13T10:12:53.878 INFO:teuthology.orchestra.console:Power on smithi106 2023-01-13T10:13:02.243 INFO:teuthology.orchestra.console:Power on for smithi043 completed 2023-01-13T10:13:02.249 INFO:teuthology.orchestra.console:Power on for smithi106 completed 2023-01-13T10:13:02.344 INFO:teuthology.provision.fog.smithi043:Waiting for deploy to finish 2023-01-13T10:13:02.350 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish 2023-01-13T10:15:36.492 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-01-13T10:15:36.494 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-01-13T10:15:45.582 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-01-13T10:15:45.582 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-01-13T10:15:54.668 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-01-13T10:15:54.669 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-01-13T10:16:03.728 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-01-13T10:16:03.729 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-01-13T10:16:12.812 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-01-13T10:16:12.813 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-01-13T10:16:21.900 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-01-13T10:16:37.324 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-01-13T10:16:46.380 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-01-13T10:16:53.227 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-13T10:16:53.545 INFO:teuthology.provision.fog.smithi043:Node is ready 2023-01-13T10:16:53.561 INFO:teuthology.orchestra.run.smithi043.stdout:smithi022 2023-01-13T10:16:53.622 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2023-01-13T10:16:54.033 INFO:teuthology.provision.fog.smithi043:Deploy complete! 2023-01-13T10:17:27.906 WARNING:teuthology.provision.fog:timed out 2023-01-13T10:17:34.157 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-13T10:17:34.463 INFO:teuthology.provision.fog.smithi106:Node is ready 2023-01-13T10:17:34.484 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106 2023-01-13T10:17:34.549 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106 2023-01-13T10:17:34.908 INFO:teuthology.provision.fog.smithi106:Deploy complete! 2023-01-13T10:17:35.141 INFO:teuthology.lock.ops:Checking smithi043.front.sepia.ceph.com 2023-01-13T10:17:35.142 INFO:teuthology.lock.ops:New key found. Updating... 2023-01-13T10:17:35.164 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com 2023-01-13T10:17:35.165 INFO:teuthology.lock.ops:New key found. Updating... 2023-01-13T10:17:35.184 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: set os type and version on server 2023-01-13T10:17:35.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-01-13T10:17:35.385 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2023-01-13T10:17:35.445 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="Red Hat Enterprise Linux" 2023-01-13T10:17:35.446 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="8.6 (Ootpa)" 2023-01-13T10:17:35.446 INFO:teuthology.orchestra.run.smithi043.stdout:ID="rhel" 2023-01-13T10:17:35.447 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="fedora" 2023-01-13T10:17:35.448 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="8.6" 2023-01-13T10:17:35.448 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el8" 2023-01-13T10:17:35.448 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-13T10:17:35.449 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2023-01-13T10:17:35.449 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-13T10:17:35.449 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://www.redhat.com/" 2023-01-13T10:17:35.449 INFO:teuthology.orchestra.run.smithi043.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-13T10:17:35.449 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-13T10:17:35.450 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-01-13T10:17:35.450 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-13T10:17:35.450 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-13T10:17:35.450 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-13T10:17:35.451 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-13T10:17:35.452 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2023-01-13T10:17:35.468 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server 2023-01-13T10:17:35.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-01-13T10:17:35.690 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2023-01-13T10:17:35.757 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Red Hat Enterprise Linux" 2023-01-13T10:17:35.758 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8.6 (Ootpa)" 2023-01-13T10:17:35.758 INFO:teuthology.orchestra.run.smithi106.stdout:ID="rhel" 2023-01-13T10:17:35.758 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="fedora" 2023-01-13T10:17:35.758 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8.6" 2023-01-13T10:17:35.758 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8" 2023-01-13T10:17:35.759 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-13T10:17:35.759 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2023-01-13T10:17:35.759 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-13T10:17:35.759 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.redhat.com/" 2023-01-13T10:17:35.759 INFO:teuthology.orchestra.run.smithi106.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-13T10:17:35.760 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-13T10:17:35.760 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-01-13T10:17:35.761 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-13T10:17:35.761 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-13T10:17:35.761 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-13T10:17:35.761 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-13T10:17:35.762 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2023-01-13T10:17:35.891 INFO:teuthology.dispatcher.supervisor:Running job 7139960 2023-01-13T10:17:35.891 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-01-12_21:12:13-rados-wip-yuri2-testing-2023-01-12-0816-distro-default-smithi/7139960 --name yuriw-2023-01-12_21:12:13-rados-wip-yuri2-testing-2023-01-12-0816-distro-default-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/yuriw-2023-01-12_21:12:13-rados-wip-yuri2-testing-2023-01-12-0816-distro-default-smithi/7139960/orig.config.yaml 2023-01-13T10:17:35.896 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-01-12_21:12:13-rados-wip-yuri2-testing-2023-01-12-0816-distro-default-smithi/7139960 2023-01-13T10:17:35.897 INFO:teuthology.dispatcher.supervisor:Job PID: 6674 2023-01-13T10:17:35.897 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-01-13T22:18:01.284 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-01-13T22:18:01.407 INFO:teuthology.kill:Killing Pids: {6674} 2023-01-13T22:18:02.247 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-13T22:18:02.247 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-13T22:18:02.248 INFO:teuthology.misc:Compressing logs... 2023-01-13T22:18:02.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-01-13T22:18:02.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-01-13T22:18:02.772 INFO:teuthology.misc:Compressing logs... 2023-01-13T22:18:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2023-01-13T22:18:04.058 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2023-01-13T22:18:04.697 INFO:teuthology.misc:Compressing logs... 2023-01-13T22:19:05.473 INFO:teuthology.kill:No teuthology processes running 2023-01-13T22:19:06.231 INFO:teuthology.kill:Nuking machines: ['smithi043', 'smithi106'] 2023-01-13T22:19:06.862 INFO:teuthology.kill:2023-01-13 22:19:06,861.861 INFO:teuthology.nuke:targets: 2023-01-13T22:19:06.863 INFO:teuthology.kill: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= 2023-01-13T22:19:06.863 INFO:teuthology.kill: smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= 2023-01-13T22:19:06.863 INFO:teuthology.kill:2023-01-13 22:19:06,862.862 INFO:teuthology.nuke:Checking targets against current locks 2023-01-13T22:19:06.929 INFO:teuthology.kill:2023-01-13 22:19:06,929.929 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-13T22:19:06.954 INFO:teuthology.kill:2023-01-13 22:19:06,954.954 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-13T22:19:07.015 INFO:teuthology.kill:2023-01-13 22:19:07,014.014 INFO:teuthology.orchestra.console:Power off smithi043 2023-01-13T22:19:07.053 INFO:teuthology.kill:2023-01-13 22:19:07,052.052 INFO:teuthology.orchestra.console:Power off smithi106 2023-01-13T22:19:15.394 INFO:teuthology.kill:2023-01-13 22:19:15,394.394 INFO:teuthology.orchestra.console:Power off for smithi043 completed 2023-01-13T22:19:15.423 INFO:teuthology.kill:2023-01-13 22:19:15,423.423 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2023-01-13T22:19:15.547 INFO:teuthology.kill:2023-01-13 22:19:15,547.547 INFO:teuthology.lock.ops:unlocked: smithi043.front.sepia.ceph.com 2023-01-13T22:19:15.570 INFO:teuthology.kill:2023-01-13 22:19:15,569.569 INFO:teuthology.lock.ops:unlocked: smithi106.front.sepia.ceph.com 2023-01-13T22:21:15.872 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-01-13T22:21:15.970 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi043 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-12_20:24:57-fs-main-distro-default-smithi/7139197 2023-01-13T22:21:15.971 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi106 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-12_20:24:57-fs-main-distro-default-smithi/7139197