2023-02-16T16:21:27.289 INFO:root:teuthology version: 0.0.1.dev41+gfbbadb5 2023-02-16T16:21:27.423 INFO:teuthology.lock.ops:Start node 'smithi162.front.sepia.ceph.com' reimaging 2023-02-16T16:21:27.424 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: reset os type and version on server 2023-02-16T16:21:27.425 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-02-16T16:21:27.464 INFO:teuthology.lock.ops:Node 'smithi162.front.sepia.ceph.com' reimaging is complete 2023-02-16T16:21:27.465 INFO:teuthology.lock.ops:Start node 'smithi170.front.sepia.ceph.com' reimaging 2023-02-16T16:21:27.465 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: reset os type and version on server 2023-02-16T16:21:27.465 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-02-16T16:21:27.482 INFO:teuthology.lock.ops:Node 'smithi170.front.sepia.ceph.com' reimaging is complete 2023-02-16T16:21:27.622 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of ubuntu 20.04 2023-02-16T16:21:27.637 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of ubuntu 20.04 2023-02-16T16:21:27.873 INFO:teuthology.orchestra.console:Power off smithi170 2023-02-16T16:21:27.889 INFO:teuthology.orchestra.console:Power off smithi162 2023-02-16T16:21:36.253 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2023-02-16T16:21:36.354 INFO:teuthology.orchestra.console:Power on smithi170 2023-02-16T16:21:40.377 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2023-02-16T16:21:40.479 INFO:teuthology.orchestra.console:Power on smithi162 2023-02-16T16:21:44.725 INFO:teuthology.orchestra.console:Power on for smithi170 completed 2023-02-16T16:21:44.731 INFO:teuthology.orchestra.console:Power on for smithi162 completed 2023-02-16T16:21:44.827 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2023-02-16T16:21:44.832 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2023-02-16T16:25:34.121 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-02-16T16:25:34.123 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-02-16T16:25:43.205 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-02-16T16:25:43.206 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-02-16T16:25:52.293 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-02-16T16:25:52.294 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-02-16T16:26:01.349 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-02-16T16:26:01.350 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-02-16T16:26:10.405 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-02-16T16:26:10.406 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-02-16T16:26:19.465 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-02-16T16:26:28.553 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-02-16T16:27:16.453 WARNING:teuthology.provision.fog:timed out 2023-02-16T16:27:22.669 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-16T16:27:23.181 INFO:teuthology.provision.fog.smithi170:Node is ready 2023-02-16T16:27:23.229 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2023-02-16T16:27:23.236 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2023-02-16T16:27:23.434 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2023-02-16T16:27:34.569 WARNING:teuthology.provision.fog:timed out 2023-02-16T16:27:40.774 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-02-16T16:27:41.252 INFO:teuthology.provision.fog.smithi162:Node is ready 2023-02-16T16:27:41.263 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com 2023-02-16T16:27:41.269 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2023-02-16T16:27:41.506 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2023-02-16T16:27:41.795 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2023-02-16T16:27:41.795 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-16T16:27:41.812 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com 2023-02-16T16:27:41.812 INFO:teuthology.lock.ops:New key found. Updating... 2023-02-16T16:27:41.830 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2023-02-16T16:27:41.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-02-16T16:27:42.012 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2023-02-16T16:27:42.059 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu" 2023-02-16T16:27:42.059 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-02-16T16:27:42.059 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu 2023-02-16T16:27:42.059 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian 2023-02-16T16:27:42.060 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-02-16T16:27:42.060 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="20.04" 2023-02-16T16:27:42.060 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/" 2023-02-16T16:27:42.060 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-02-16T16:27:42.060 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-02-16T16:27:42.060 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-02-16T16:27:42.060 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=focal 2023-02-16T16:27:42.061 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=focal 2023-02-16T16:27:42.061 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-02-16T16:27:42.077 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server 2023-02-16T16:27:42.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-02-16T16:27:42.341 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2023-02-16T16:27:42.347 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2023-02-16T16:27:42.348 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-02-16T16:27:42.348 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2023-02-16T16:27:42.348 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2023-02-16T16:27:42.348 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-02-16T16:27:42.348 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="20.04" 2023-02-16T16:27:42.348 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2023-02-16T16:27:42.348 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-02-16T16:27:42.349 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-02-16T16:27:42.349 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-02-16T16:27:42.349 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=focal 2023-02-16T16:27:42.349 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=focal 2023-02-16T16:27:42.349 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-02-16T16:27:42.439 INFO:teuthology.dispatcher.supervisor:Running job 7176328 2023-02-16T16:27:42.439 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-02-16_16:16:31-rados-wip-yuri7-testing-2023-02-08-1414-distro-default-smithi/7176328 --name yuriw-2023-02-16_16:16:31-rados-wip-yuri7-testing-2023-02-08-1414-distro-default-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} -- /home/teuthworker/archive/yuriw-2023-02-16_16:16:31-rados-wip-yuri7-testing-2023-02-08-1414-distro-default-smithi/7176328/orig.config.yaml 2023-02-16T16:27:42.443 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-02-16_16:16:31-rados-wip-yuri7-testing-2023-02-08-1414-distro-default-smithi/7176328 2023-02-16T16:27:42.444 INFO:teuthology.dispatcher.supervisor:Job PID: 18619 2023-02-16T16:27:42.444 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-02-17T04:28:40.545 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-02-17T04:28:43.524 INFO:teuthology.kill:Killing Pids: {18619} 2023-02-17T04:28:44.656 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-02-17T04:28:44.657 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-02-17T04:28:44.663 INFO:teuthology.misc:Compressing logs... 2023-02-17T04:28:44.692 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-02-17T04:28:46.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-02-17T04:28:46.916 INFO:teuthology.misc:Compressing logs... 2023-02-17T04:29:23.115 INFO:teuthology.misc:Compressing logs... 2023-02-17T04:29:40.934 INFO:teuthology.kill:No teuthology processes running 2023-02-17T04:29:42.046 INFO:teuthology.kill:Nuking machines: ['smithi162', 'smithi170'] 2023-02-17T04:29:42.643 INFO:teuthology.kill:2023-02-17 04:29:42,643.643 INFO:teuthology.nuke:targets: 2023-02-17T04:29:42.646 INFO:teuthology.kill: smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= 2023-02-17T04:29:42.646 INFO:teuthology.kill: smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= 2023-02-17T04:29:42.646 INFO:teuthology.kill:2023-02-17 04:29:42,643.643 INFO:teuthology.nuke:Checking targets against current locks 2023-02-17T04:29:42.691 INFO:teuthology.kill:2023-02-17 04:29:42,690.690 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-17T04:29:42.713 INFO:teuthology.kill:2023-02-17 04:29:42,713.713 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-02-17T04:29:42.780 INFO:teuthology.kill:2023-02-17 04:29:42,780.780 INFO:teuthology.orchestra.console:Power off smithi170 2023-02-17T04:29:42.794 INFO:teuthology.kill:2023-02-17 04:29:42,793.793 INFO:teuthology.orchestra.console:Power off smithi162 2023-02-17T04:29:51.151 INFO:teuthology.kill:2023-02-17 04:29:51,151.151 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2023-02-17T04:29:51.301 INFO:teuthology.kill:2023-02-17 04:29:51,301.301 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com 2023-02-17T04:29:55.270 INFO:teuthology.kill:2023-02-17 04:29:55,270.270 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2023-02-17T04:29:55.483 INFO:teuthology.kill:2023-02-17 04:29:55,483.483 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com 2023-02-17T04:31:55.860 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-02-17T04:31:56.047 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi162 but it was locked by another job: /home/teuthworker/archive/dis-2023-02-17_00:48:46-rbd-wip-dis-testing-distro-default-smithi/7178024 2023-02-17T04:31:56.048 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi170 but it was locked by another job: /home/teuthworker/archive/dis-2023-02-17_00:48:46-rbd-wip-dis-testing-distro-default-smithi/7178024