2022-04-05T06:35:27.912 INFO:root:teuthology version: 1.1.0-d877b023 2022-04-05T06:35:28.070 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging 2022-04-05T06:35:28.071 INFO:teuthology.lock.ops:Updating [ubuntu@smithi082.front.sepia.ceph.com]: reset os type and version on server 2022-04-05T06:35:28.072 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2022-04-05T06:35:28.091 INFO:teuthology.lock.ops:Node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging is complete 2022-04-05T06:35:28.092 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging 2022-04-05T06:35:28.092 INFO:teuthology.lock.ops:Updating [ubuntu@smithi133.front.sepia.ceph.com]: reset os type and version on server 2022-04-05T06:35:28.093 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-04-05T06:35:28.112 INFO:teuthology.lock.ops:Node 'ubuntu@smithi133.front.sepia.ceph.com' reimaging is complete 2022-04-05T06:35:28.113 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging 2022-04-05T06:35:28.113 INFO:teuthology.lock.ops:Updating [ubuntu@smithi165.front.sepia.ceph.com]: reset os type and version on server 2022-04-05T06:35:28.114 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2022-04-05T06:35:28.134 INFO:teuthology.lock.ops:Node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging is complete 2022-04-05T06:35:28.304 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of centos 8.stream 2022-04-05T06:35:28.331 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of centos 8.stream 2022-04-05T06:35:28.358 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of centos 8.stream 2022-04-05T06:35:28.720 INFO:teuthology.orchestra.console:Power off smithi133 2022-04-05T06:35:28.763 INFO:teuthology.orchestra.console:Power off smithi082 2022-04-05T06:35:28.799 INFO:teuthology.orchestra.console:Power off smithi165 2022-04-05T06:35:37.097 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-04-05T06:35:37.131 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2022-04-05T06:35:37.165 INFO:teuthology.orchestra.console:Power off for smithi165 completed 2022-04-05T06:35:37.199 INFO:teuthology.orchestra.console:Power on smithi133 2022-04-05T06:35:37.232 INFO:teuthology.orchestra.console:Power on smithi082 2022-04-05T06:35:37.267 INFO:teuthology.orchestra.console:Power on smithi165 2022-04-05T06:35:45.567 INFO:teuthology.orchestra.console:Power on for smithi133 completed 2022-04-05T06:35:45.596 INFO:teuthology.orchestra.console:Power on for smithi082 completed 2022-04-05T06:35:45.669 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish 2022-04-05T06:35:45.698 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish 2022-04-05T06:35:45.719 INFO:teuthology.orchestra.console:Power on for smithi165 completed 2022-04-05T06:35:45.820 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish 2022-04-05T06:39:49.176 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-05T06:40:09.583 INFO:teuthology.provision.fog.smithi082:Node is ready 2022-04-05T06:40:09.690 INFO:teuthology.orchestra.run.smithi082.stdout:smithi103 2022-04-05T06:40:09.747 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.103 smithi103.front.sepia.ceph.com smithi103 2022-04-05T06:40:10.110 INFO:teuthology.provision.fog.smithi082:Deploy complete! 2022-04-05T06:40:23.911 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-05T06:40:24.177 INFO:teuthology.provision.fog.smithi165:Node is ready 2022-04-05T06:40:24.195 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com 2022-04-05T06:40:24.252 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165 2022-04-05T06:40:24.596 INFO:teuthology.provision.fog.smithi165:Deploy complete! 2022-04-05T06:40:59.966 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-05T06:41:00.226 INFO:teuthology.provision.fog.smithi133:Node is ready 2022-04-05T06:41:00.244 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com 2022-04-05T06:41:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2022-04-05T06:41:00.652 INFO:teuthology.provision.fog.smithi133:Deploy complete! 2022-04-05T06:41:00.869 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com 2022-04-05T06:41:00.870 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-05T06:41:00.889 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com 2022-04-05T06:41:00.889 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-05T06:41:00.907 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com 2022-04-05T06:41:00.908 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-05T06:41:00.929 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server 2022-04-05T06:41:00.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2022-04-05T06:41:01.187 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2022-04-05T06:41:01.244 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2022-04-05T06:41:01.245 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8" 2022-04-05T06:41:01.245 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2022-04-05T06:41:01.245 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2022-04-05T06:41:01.245 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8" 2022-04-05T06:41:01.246 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2022-04-05T06:41:01.246 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 8" 2022-04-05T06:41:01.246 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2022-04-05T06:41:01.246 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-04-05T06:41:01.247 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2022-04-05T06:41:01.247 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-04-05T06:41:01.247 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-04-05T06:41:01.247 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-04-05T06:41:01.249 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2022-04-05T06:41:01.268 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server 2022-04-05T06:41:01.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-04-05T06:41:01.459 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2022-04-05T06:41:01.516 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2022-04-05T06:41:01.516 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8" 2022-04-05T06:41:01.517 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2022-04-05T06:41:01.517 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2022-04-05T06:41:01.517 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8" 2022-04-05T06:41:01.517 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2022-04-05T06:41:01.518 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 8" 2022-04-05T06:41:01.518 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2022-04-05T06:41:01.518 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-04-05T06:41:01.518 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2022-04-05T06:41:01.519 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-04-05T06:41:01.519 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-04-05T06:41:01.519 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-04-05T06:41:01.520 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-04-05T06:41:01.539 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server 2022-04-05T06:41:01.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2022-04-05T06:41:01.781 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2022-04-05T06:41:01.839 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2022-04-05T06:41:01.839 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="8" 2022-04-05T06:41:01.842 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2022-04-05T06:41:01.842 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2022-04-05T06:41:01.843 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="8" 2022-04-05T06:41:01.843 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el8" 2022-04-05T06:41:01.843 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 8" 2022-04-05T06:41:01.843 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2022-04-05T06:41:01.844 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-04-05T06:41:01.844 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2022-04-05T06:41:01.844 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-04-05T06:41:01.844 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-04-05T06:41:01.844 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-04-05T06:41:01.845 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2022-04-05T06:41:02.065 INFO:teuthology.dispatcher.supervisor:Running job 6777261 2022-04-05T06:41:02.065 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-04-05_04:17:02-fs-pacific-distro-default-smithi/6777261 --name teuthology-2022-04-05_04:17:02-fs-pacific-distro-default-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} -- /home/teuthworker/archive/teuthology-2022-04-05_04:17:02-fs-pacific-distro-default-smithi/6777261/orig.config.yaml 2022-04-05T06:41:02.069 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-04-05_04:17:02-fs-pacific-distro-default-smithi/6777261 2022-04-05T06:41:02.070 INFO:teuthology.dispatcher.supervisor:Job PID: 24593 2022-04-05T06:41:02.070 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-04-05T13:11:30.857 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-04-05T13:11:31.026 INFO:teuthology.kill:Killing Pids: {24593} 2022-04-05T13:11:32.220 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2022-04-05T13:11:32.220 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2022-04-05T13:11:32.221 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2022-04-05T13:11:32.221 INFO:teuthology.misc:Compressing logs... 2022-04-05T13:11:32.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2022-04-05T13:11:32.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-04-05T13:11:32.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2022-04-05T13:11:32.973 INFO:teuthology.misc:Compressing logs... 2022-04-05T13:11:33.234 INFO:teuthology.misc:Compressing logs... 2022-04-05T13:16:03.056 INFO:teuthology.kill:No teuthology processes running 2022-04-05T13:16:04.220 INFO:teuthology.kill:Nuking machines: ['smithi082', 'smithi133', 'smithi165'] 2022-04-05T13:16:05.243 INFO:teuthology.kill:2022-04-05 13:16:05,243.243 INFO:teuthology.nuke:targets: 2022-04-05T13:16:05.244 INFO:teuthology.kill: smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwfgpuVvGaWATZXNbpQSKIT4534U//CkA8LH40h96ro0qOzM0NZ+RC/teLimbpPLFy6yIgGMkvSw1TU8QsXhYjuFpx3WX4gCVtkg2LYh1nCQx2CKzbkqltD/CAvStWkjCBnp7TSQt1jKbjst6tOKR3ANXOpEr0MzunKVHnVp85fXpyvUJHg1/I6+98raOIJ8MFredrhorlKmcGveZbuIpL+uk4N54OxcJwziNJxIbKxEgkVq42pNb5ZQYL7712s684P/EdmTlHaZmDXvO4yLTY3MQ5MqjA6EUI2cAJFN5PHV+j5CaqsX7VhKy2Ooe4b+1kLa+4C9MyY5cFc49wWB5T8mTzqubK7Xk09rUqaUNLLMS5CV8cIN9K1SezNsWmntu3hJlZEWA03uuA4xpNjgXStAAUhl3s1Kjinc3BZupsasKe4nD8nZRi6zwGz+5fdgr7nqkKRmBi6lWDYxqXHa79+VAv4e+JzIvUxCPFIylD3/1luUPWN20qyiGZF1392gk= 2022-04-05T13:16:05.245 INFO:teuthology.kill: smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+tr3pXscg3k5XDhrnQZD01mDxZC04YgQ8PVbngGb6QUbSzz/1mFFSSt709tytFVmrqJmPodXrwGI5G4Tu//ZpOFV1MJTUXLCZSjsHz9cROVy/hr5JzCyFw3G/vORr76n85DkWVIODBvMjYk1ciTiiZyE90MJ/rhwfqY0xspL3kQii5Goh/eHrro2CSXADxhpBqjXtvqgyAHVNcyXv4W1XPm8tLOMOUrhZaCHdQQtRjFSo1dfSWH7uHJpKcAd1+/AqVrTF0m6sEP6oy4GYYDC59E87thyP4f8WC1RZzjs2Nt+avjVawiZbCXJZYJ+eFzNIlWn05ZogIB1v+0hLbQ3XGNoFIBUrbnd1IW1yGH+7wq4imlrEzne69D6oxUgR3WvBHFH8q/r4RkcTgKysvn8SCfMUHA7oJXMphlBcVvC/Q8Wd8pqBMob+tKkISkSDF/U5vXZqGNy/DqGQ2sKtI245PWlTHVesRIQZLG1KK/2Bpa1pP9vWiuy+eTUR9BpVg2k= 2022-04-05T13:16:05.245 INFO:teuthology.kill: smithi165.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5wx0JMkOeEOB1Fk+pwVTfExdxpSKCcHGpB7Zg5XQsWnAKJ3ofW4fdh7WV/x0Qu7iPI0s4NonTgJZi4llTuXjvJ+xUJLdZg0UBdmwQNssoAJ/rfsH7M/aTSlejfumLD5Vg+z6eoiUUmxiacSo6GW5MyY8KrdPLhg4a4IgpHlfu9Ufp+YB1pGHX53/yaHWK7+ypLI5N5KxLvoE7AMMD7fAGB94Z6GDrBMSdcysfrU/DFwGg7NvKP9Nuwrxz8VYxxnM1G9l/oIrsgB1OK257PPLrCIXKn8MiOWPXv6OAPn8q6k1FHIkLc9BgCQhLXWByCMEv9LRLOOpKBZHWnUd6caJPoNkkPNrAs07mxsOyzn3/XEpRrJAGA3C0DxPe7Gf7LEB/IQ7qToNF9W+/4OgtAenzyCL8tjSKIAfoES8qcBxF28u/qCqM9bJScbCXcOI/rYmT88J8JngkPQl0cPjc1o3G2v76GpGByKeqtT0I4bNCMjZV6T8TA+V9XH4JXH14ie0= 2022-04-05T13:16:05.276 INFO:teuthology.kill:2022-04-05 13:16:05,276.276 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-05T13:16:05.281 INFO:teuthology.kill:2022-04-05 13:16:05,281.281 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-05T13:16:05.283 INFO:teuthology.kill:2022-04-05 13:16:05,282.282 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-05T13:16:05.368 INFO:teuthology.kill:2022-04-05 13:16:05,367.367 INFO:teuthology.orchestra.console:Power off smithi082 2022-04-05T13:16:05.383 INFO:teuthology.kill:2022-04-05 13:16:05,383.383 INFO:teuthology.orchestra.console:Power off smithi133 2022-04-05T13:16:05.394 INFO:teuthology.kill:2022-04-05 13:16:05,394.394 INFO:teuthology.orchestra.console:Power off smithi165 2022-04-05T13:16:13.756 INFO:teuthology.kill:2022-04-05 13:16:13,756.756 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2022-04-05T13:16:13.762 INFO:teuthology.kill:2022-04-05 13:16:13,762.762 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2022-04-05T13:16:13.768 INFO:teuthology.kill:2022-04-05 13:16:13,768.768 INFO:teuthology.orchestra.console:Power off for smithi165 completed 2022-04-05T13:16:13.910 INFO:teuthology.kill:2022-04-05 13:16:13,909.909 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com 2022-04-05T13:16:13.914 INFO:teuthology.kill:2022-04-05 13:16:13,914.914 INFO:teuthology.lock.ops:unlocked: smithi133.front.sepia.ceph.com 2022-04-05T13:16:13.918 INFO:teuthology.kill:2022-04-05 13:16:13,917.917 INFO:teuthology.lock.ops:unlocked: smithi165.front.sepia.ceph.com 2022-04-05T13:18:14.255 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-04-05T13:18:14.399 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi082 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-05_03:15:03-fs-quincy-distro-default-smithi/6776890 2022-04-05T13:18:14.400 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi133 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-05_03:15:03-fs-quincy-distro-default-smithi/6776890 2022-04-05T13:18:14.400 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi165 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-05_03:15:03-fs-quincy-distro-default-smithi/6776890