2022-02-07T15:01:47.333 INFO:root:teuthology version: 1.1.0-3094160c 2022-02-07T15:01:47.455 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi005.front.sepia.ceph.com' reimaging 2022-02-07T15:01:47.456 INFO:teuthology.lock.ops:Updating [ubuntu@smithi005.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T15:01:47.456 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2022-02-07T15:01:47.474 INFO:teuthology.lock.ops:Node 'ubuntu@smithi005.front.sepia.ceph.com' reimaging is complete 2022-02-07T15:01:47.475 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi122.front.sepia.ceph.com' reimaging 2022-02-07T15:01:47.475 INFO:teuthology.lock.ops:Updating [ubuntu@smithi122.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T15:01:47.475 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2022-02-07T15:01:47.494 INFO:teuthology.lock.ops:Node 'ubuntu@smithi122.front.sepia.ceph.com' reimaging is complete 2022-02-07T15:01:47.710 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of ubuntu 20.04 2022-02-07T15:01:47.723 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of ubuntu 20.04 2022-02-07T15:01:48.116 INFO:teuthology.orchestra.console:Power off smithi005 2022-02-07T15:01:48.130 INFO:teuthology.orchestra.console:Power off smithi122 2022-02-07T15:01:48.260 INFO:teuthology.orchestra.console:Power off for smithi005 completed 2022-02-07T15:01:48.272 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2022-02-07T15:01:48.361 INFO:teuthology.orchestra.console:Power on smithi005 2022-02-07T15:01:48.373 INFO:teuthology.orchestra.console:Power on smithi122 2022-02-07T15:01:52.619 INFO:teuthology.orchestra.console:Power on for smithi005 completed 2022-02-07T15:01:52.721 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish 2022-02-07T15:01:56.738 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2022-02-07T15:01:56.839 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2022-02-07T15:06:30.991 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T15:06:31.537 INFO:teuthology.provision.fog.smithi005:Node is ready 2022-02-07T15:06:31.587 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005.front.sepia.ceph.com 2022-02-07T15:06:31.594 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5 smithi005.front.sepia.ceph.com smithi005 2022-02-07T15:06:31.716 INFO:teuthology.provision.fog.smithi005:Deploy complete! 2022-02-07T15:06:51.309 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T15:06:51.770 INFO:teuthology.provision.fog.smithi122:Node is ready 2022-02-07T15:06:51.777 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122.front.sepia.ceph.com 2022-02-07T15:06:51.827 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122 smithi122.front.sepia.ceph.com smithi122 2022-02-07T15:06:52.026 INFO:teuthology.provision.fog.smithi122:Deploy complete! 2022-02-07T15:06:52.208 INFO:teuthology.lock.ops:Checking smithi005.front.sepia.ceph.com 2022-02-07T15:06:52.209 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T15:06:52.228 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com 2022-02-07T15:06:52.229 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T15:06:52.247 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: set os type and version on server 2022-02-07T15:06:52.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2022-02-07T15:06:52.550 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2022-02-07T15:06:52.557 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Ubuntu" 2022-02-07T15:06:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-07T15:06:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:ID=ubuntu 2022-02-07T15:06:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE=debian 2022-02-07T15:06:52.559 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-07T15:06:52.560 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="20.04" 2022-02-07T15:06:52.560 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-07T15:06:52.560 INFO:teuthology.orchestra.run.smithi005.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-07T15:06:52.560 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-07T15:06:52.560 INFO:teuthology.orchestra.run.smithi005.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-07T15:06:52.561 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_CODENAME=focal 2022-02-07T15:06:52.561 INFO:teuthology.orchestra.run.smithi005.stdout:UBUNTU_CODENAME=focal 2022-02-07T15:06:52.562 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2022-02-07T15:06:52.580 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server 2022-02-07T15:06:52.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2022-02-07T15:06:52.750 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2022-02-07T15:06:52.757 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2022-02-07T15:06:52.757 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-07T15:06:52.758 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2022-02-07T15:06:52.758 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2022-02-07T15:06:52.758 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-07T15:06:52.759 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="20.04" 2022-02-07T15:06:52.759 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-07T15:06:52.760 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-07T15:06:52.760 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=focal 2022-02-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=focal 2022-02-07T15:06:52.762 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2022-02-07T15:06:52.857 INFO:teuthology.dispatcher.supervisor:Running job 6666493 2022-02-07T15:06:52.857 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666493 --name benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} -- /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666493/orig.config.yaml 2022-02-07T15:06:52.861 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666493 2022-02-07T15:06:52.861 INFO:teuthology.dispatcher.supervisor:Job PID: 24317 2022-02-07T15:06:52.862 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-07T21:37:21.424 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-07T21:37:21.598 INFO:teuthology.kill:Killing Pids: {24317} 2022-02-07T21:37:22.809 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-07T21:37:22.810 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-07T21:37:22.810 INFO:teuthology.misc:Compressing logs... 2022-02-07T21:37:22.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2022-02-07T21:37:22.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2022-02-07T21:37:23.228 INFO:teuthology.misc:Compressing logs... 2022-02-07T21:37:24.106 INFO:teuthology.misc:Compressing logs... 2022-02-07T21:37:25.706 INFO:teuthology.kill:No teuthology processes running 2022-02-07T21:37:26.897 INFO:teuthology.kill:Nuking machines: ['smithi005', 'smithi122'] 2022-02-07T21:37:27.845 INFO:teuthology.kill:2022-02-07 21:37:27,845.845 INFO:teuthology.nuke:targets: 2022-02-07T21:37:27.846 INFO:teuthology.kill: smithi005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX7FEWXILbIfXUWTti/0wbWsblOQ7n5eyU88U04fDLunFtEdxKWxZe32zRNtzjmBtnShPsf+/6V8y2NWZ5tOyFlzvQrpeSz9yb268L0wszL9nQdeqzpVDWlym+2GJLrwdFna4+TAWEdBjIjZ6XGFY0zYjl+b58bQjfYAcn77dgmI7ZI50p9hXGnGAc6jDUIVTNjTAePVbWVvol+HhmVj7UtqnrXFJ0EQZqGoYYKjZB3PTCqifblrbpdR9f1QOwY7FJ9vEAAOiD+1DJQ6PPO8m1WDE6XTyfL33ArjaG0ovIsAUVfBKVpez1DnywhTBrYU19yg7vMPW+pourtJYwVOWRRLDIBZ/+2bmyiYfsv4GDOfLSeG9xx3YPgz2i+T69WH74lgEvkpdvlO48bSXBWcUc8j8gA8aXTXnVyuapremQ8ckcih2g7+F4hKUEuNJHMh33UsON78RzObdNqzvfOWUUnlauMdi7JXC6i7ERp9uNiEzxAEqmrnHbTtXiuH87PDs= 2022-02-07T21:37:27.846 INFO:teuthology.kill: smithi122.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWcK+ZDydDj7AuFLtB2bGIBumWnJHdtTGVT9luCABCtdfwkHHinn7UMaBB9/TTiPNQamdhA9v/NlEeNxuODO+mlChiUgfaEDpFgYDKCy6fZAf13iwF9/7pYrGATMSGcpQLyLeWHm6kfw3tYMnPnN6nz1PcyJa/Ri8+1FmJwBj0avq1WTNQO599k5vOfGwLVAHoxEQmLTAMh+8lYetjgBfMdgqkZn/BJD1VP8a9osePn6T3igNkgfdzvPTMy+uYkfbHCpyii+9AWFNhMn6X3ygn9DeDHBaPKhc67+DZ8x7DHkUUzaYbfUJlza01fbr0qotXvmaEJfZ1So1OH0WRsR2lZWFcmRH2xid59dGE1LIBZ1n5i/AUCfvpyJFo3bpDouiu89mwX2VpFG61GVdHmoBn+mayozOEG3iQ2cez3FhDNJDsTAFA0YNQZGO2kWa9Q2XO9Q17LSowSj+VSxU1h54e0R9t60BMyqjFnqFF28KGUpSd4HOX0AykiPWN1EU7ej8= 2022-02-07T21:37:27.872 INFO:teuthology.kill:2022-02-07 21:37:27,872.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T21:37:27.875 INFO:teuthology.kill:2022-02-07 21:37:27,875.875 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T21:37:27.952 INFO:teuthology.kill:2022-02-07 21:37:27,952.952 INFO:teuthology.orchestra.console:Power off smithi005 2022-02-07T21:37:27.966 INFO:teuthology.kill:2022-02-07 21:37:27,966.966 INFO:teuthology.orchestra.console:Power off smithi122 2022-02-07T21:37:36.331 INFO:teuthology.kill:2022-02-07 21:37:36,330.330 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2022-02-07T21:37:36.480 INFO:teuthology.kill:2022-02-07 21:37:36,479.479 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com 2022-02-07T21:37:40.435 INFO:teuthology.kill:2022-02-07 21:37:40,434.434 INFO:teuthology.orchestra.console:Power off for smithi005 completed 2022-02-07T21:37:40.581 INFO:teuthology.kill:2022-02-07 21:37:40,581.581 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com 2022-02-07T21:39:40.906 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-07T21:39:41.014 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi005 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6666848 2022-02-07T21:39:41.014 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi122 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6666842