2023-02-11T06:54:05.652 INFO:root:teuthology version: 0.0.1.dev37+g00690ec 2023-02-11T06:54:05.982 INFO:teuthology.lock.ops:Start node 'smithi142.front.sepia.ceph.com' reimaging 2023-02-11T06:54:06.053 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: reset os type and version on server 2023-02-11T06:54:06.053 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2023-02-11T06:54:06.293 INFO:teuthology.lock.ops:Node 'smithi142.front.sepia.ceph.com' reimaging is complete 2023-02-11T06:54:06.293 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging 2023-02-11T06:54:06.294 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server 2023-02-11T06:54:06.294 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-02-11T06:54:06.381 INFO:teuthology.lock.ops:Node 'smithi169.front.sepia.ceph.com' reimaging is complete 2023-02-11T06:54:06.437 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of centos 8.stream 2023-02-11T06:54:06.557 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 8.stream 2023-02-11T06:54:06.738 INFO:teuthology.orchestra.console:Power off smithi142 2023-02-11T06:54:06.776 INFO:teuthology.orchestra.console:Power off smithi169 2023-02-11T06:54:15.231 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2023-02-11T06:54:15.433 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2023-02-11T06:54:15.534 INFO:teuthology.orchestra.console:Power on smithi169 2023-02-11T06:54:15.635 INFO:teuthology.orchestra.console:Power on smithi142 2023-02-11T06:54:19.883 INFO:teuthology.orchestra.console:Power on for smithi169 completed 2023-02-11T06:54:19.951 INFO:teuthology.orchestra.console:Power on for smithi142 completed 2023-02-11T06:54:20.052 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2023-02-11T06:54:20.134 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish 2023-02-11T06:57:24.485 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-02-11T06:57:33.605 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-02-11T06:57:39.593 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-02-11T06:57:42.693 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-02-11T06:57:48.681 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-02-11T06:57:51.749 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-02-11T06:57:57.765 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-02-11T06:58:06.857 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-02-11T06:58:15.941 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-02-11T06:58:57.790 WARNING:teuthology.provision.fog:timed out 2023-02-11T06:59:04.092 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-11T06:59:04.356 INFO:teuthology.provision.fog.smithi142:Node is ready 2023-02-11T06:59:04.417 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com 2023-02-11T06:59:04.476 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142 2023-02-11T06:59:04.846 INFO:teuthology.provision.fog.smithi142:Deploy complete! 2023-02-11T06:59:21.975 WARNING:teuthology.provision.fog:timed out 2023-02-11T06:59:28.343 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-11T06:59:28.604 INFO:teuthology.provision.fog.smithi169:Node is ready 2023-02-11T06:59:28.663 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2023-02-11T06:59:28.852 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2023-02-11T06:59:29.233 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2023-02-11T06:59:29.432 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com 2023-02-11T06:59:29.433 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-11T06:59:29.450 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2023-02-11T06:59:29.450 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-11T06:59:29.469 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server 2023-02-11T06:59:29.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-02-11T06:59:29.719 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2023-02-11T06:59:29.775 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 8" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-11T06:59:29.776 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-02-11T06:59:29.777 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2023-02-11T06:59:29.795 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2023-02-11T06:59:29.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-02-11T06:59:30.041 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2023-02-11T06:59:30.100 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2023-02-11T06:59:30.100 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8" 2023-02-11T06:59:30.100 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2023-02-11T06:59:30.100 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2023-02-11T06:59:30.100 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8" 2023-02-11T06:59:30.100 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8" 2023-02-11T06:59:30.100 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 8" 2023-02-11T06:59:30.101 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2023-02-11T06:59:30.101 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-02-11T06:59:30.101 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2023-02-11T06:59:30.101 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-02-11T06:59:30.101 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-02-11T06:59:30.101 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-02-11T06:59:30.101 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-02-11T06:59:30.196 INFO:teuthology.dispatcher.supervisor:Running job 7160275 2023-02-11T06:59:30.197 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_00690eca1aa0bd56527bb28e9f9e35d734d406bf/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160275 --name yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi --description rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/failover} -- /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160275/orig.config.yaml 2023-02-11T06:59:30.200 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-02-08_19:35:19-rados-wip-yuri11-testing-2023-02-06-1424-distro-default-smithi/7160275 2023-02-11T06:59:30.201 INFO:teuthology.dispatcher.supervisor:Job PID: 12903 2023-02-11T06:59:30.201 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-02-11T19:00:32.717 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-02-11T19:00:37.276 INFO:teuthology.kill:Killing Pids: {12903} 2023-02-11T19:00:45.992 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2023-02-11T19:00:45.992 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2023-02-11T19:00:46.030 INFO:teuthology.misc:Compressing logs... 2023-02-11T19:00:46.068 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-02-11T19:00:48.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-02-11T19:00:49.767 INFO:teuthology.kill:No teuthology processes running 2023-02-11T19:00:50.862 INFO:teuthology.kill:Nuking machines: ['smithi142', 'smithi169'] 2023-02-11T19:00:51.960 INFO:teuthology.kill:2023-02-11 19:00:51,960.960 INFO:teuthology.nuke:targets: 2023-02-11T19:00:51.977 INFO:teuthology.kill: smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP9sGaPRunoeYogMdOLXe1nN1LgASfMOyeldr5kxA3DZ1iuXPQFI0+rIBxlKRUQiJkfdMOnBAXAUd3T8XaxrbQ4= 2023-02-11T19:00:51.978 INFO:teuthology.kill: smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDleA5I54OnJStOMbnLt7nP+QaIgZV1nt4nRQRYU1jnW3RsC0eX04xyf+0lwYMe9WHtS6kkzN/eVTyciESfHMe1iQp+E0BhPXZknnNxLBalqdYHKTgUmPj1pwQOdRNg0MyI24ma3TIxvFVrJtJzQingYgSokRlCbl4ED87YDM9Ek8StHUA5mdpyVZjToH0LTLvlQ1Hgr6dyXqhBAywx+S4aaKQ07cjbvScY8Q4aWb6jq4jmwXk+uOkoJTrV3i2OUA+IM0e4ksQ20wSjdu39sOU0BdB/k/SJYaejD++YgmZyyhybKAz3OnXqSyEwXq0x8gUfaDnQCCKdccrdw7ElQ6UDl/NnggR0W1beEHo3q8rpao9rdVlWPWtnXtesI0pUCwq44K+Yb67RJoacBz7L0gfwADNHsIx6OeMt8Kdq9wP36+fN3jn2+Gyq45ILj3fIkvuQkAU62pSVHDWe8oYd+j/NhUOBvpQPT/boWH3+pmSVjcN2SQAmcVnZ4aZhr3pD9bM= 2023-02-11T19:00:51.980 INFO:teuthology.kill:2023-02-11 19:00:51,960.960 INFO:teuthology.nuke:Checking targets against current locks 2023-02-11T19:00:52.009 INFO:teuthology.kill:2023-02-11 19:00:52,009.009 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-11T19:00:52.045 INFO:teuthology.kill:2023-02-11 19:00:52,044.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-11T19:00:52.078 INFO:teuthology.kill:2023-02-11 19:00:52,077.077 INFO:teuthology.orchestra.console:Power off smithi142 2023-02-11T19:00:52.119 INFO:teuthology.kill:2023-02-11 19:00:52,119.119 INFO:teuthology.orchestra.console:Power off smithi169 2023-02-11T19:01:00.571 INFO:teuthology.kill:2023-02-11 19:01:00,570.570 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2023-02-11T19:01:00.721 INFO:teuthology.kill:2023-02-11 19:01:00,720.720 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com 2023-02-11T19:01:04.691 INFO:teuthology.kill:2023-02-11 19:01:04,690.690 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2023-02-11T19:01:04.840 INFO:teuthology.kill:2023-02-11 19:01:04,840.840 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com 2023-02-11T19:03:05.220 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-02-11T19:03:05.444 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi142 but it was locked by another job: /home/teuthworker/archive/cbodley-2023-02-11_16:16:51-rgw-wip-cbodley-testing-distro-default-smithi/7169635 2023-02-11T19:03:05.444 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi169 but it was locked by another job: /home/teuthworker/archive/cbodley-2023-02-11_16:16:51-rgw-wip-cbodley-testing-distro-default-smithi/7169634