2022-02-06T19:19:16.258 INFO:root:teuthology version: 1.1.0-3094160c 2022-02-06T19:19:16.400 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging 2022-02-06T19:19:16.402 INFO:teuthology.lock.ops:Updating [ubuntu@smithi082.front.sepia.ceph.com]: reset os type and version on server 2022-02-06T19:19:16.402 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2022-02-06T19:19:16.423 INFO:teuthology.lock.ops:Node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging is complete 2022-02-06T19:19:16.423 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging 2022-02-06T19:19:16.424 INFO:teuthology.lock.ops:Updating [ubuntu@smithi084.front.sepia.ceph.com]: reset os type and version on server 2022-02-06T19:19:16.424 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2022-02-06T19:19:16.448 INFO:teuthology.lock.ops:Node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging is complete 2022-02-06T19:19:16.636 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of ubuntu 20.04 2022-02-06T19:19:16.661 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of ubuntu 20.04 2022-02-06T19:19:16.957 INFO:teuthology.orchestra.console:Power off smithi082 2022-02-06T19:19:16.997 INFO:teuthology.orchestra.console:Power off smithi084 2022-02-06T19:19:25.361 INFO:teuthology.orchestra.console:Power off for smithi084 completed 2022-02-06T19:19:25.462 INFO:teuthology.orchestra.console:Power on smithi084 2022-02-06T19:19:29.439 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2022-02-06T19:19:29.541 INFO:teuthology.orchestra.console:Power on smithi082 2022-02-06T19:19:33.824 INFO:teuthology.orchestra.console:Power on for smithi084 completed 2022-02-06T19:19:33.926 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish 2022-02-06T19:19:37.906 INFO:teuthology.orchestra.console:Power on for smithi082 completed 2022-02-06T19:19:38.008 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish 2022-02-06T19:24:30.699 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-06T19:24:31.141 INFO:teuthology.provision.fog.smithi082:Node is ready 2022-02-06T19:24:31.148 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082.front.sepia.ceph.com 2022-02-06T19:24:31.154 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082 2022-02-06T19:24:31.369 INFO:teuthology.provision.fog.smithi082:Deploy complete! 2022-02-06T19:24:37.070 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-06T19:24:37.480 INFO:teuthology.provision.fog.smithi084:Node is ready 2022-02-06T19:24:37.486 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com 2022-02-06T19:24:37.493 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084 2022-02-06T19:24:37.700 INFO:teuthology.provision.fog.smithi084:Deploy complete! 2022-02-06T19:24:37.926 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com 2022-02-06T19:24:37.927 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-06T19:24:37.946 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com 2022-02-06T19:24:37.946 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-06T19:24:37.964 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server 2022-02-06T19:24:37.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2022-02-06T19:24:38.185 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2022-02-06T19:24:38.192 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Ubuntu" 2022-02-06T19:24:38.193 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-06T19:24:38.193 INFO:teuthology.orchestra.run.smithi082.stdout:ID=ubuntu 2022-02-06T19:24:38.193 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE=debian 2022-02-06T19:24:38.193 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-06T19:24:38.194 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="20.04" 2022-02-06T19:24:38.194 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-06T19:24:38.194 INFO:teuthology.orchestra.run.smithi082.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-06T19:24:38.194 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-06T19:24:38.195 INFO:teuthology.orchestra.run.smithi082.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-06T19:24:38.195 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_CODENAME=focal 2022-02-06T19:24:38.195 INFO:teuthology.orchestra.run.smithi082.stdout:UBUNTU_CODENAME=focal 2022-02-06T19:24:38.196 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2022-02-06T19:24:38.215 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server 2022-02-06T19:24:38.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2022-02-06T19:24:38.379 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2022-02-06T19:24:38.386 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Ubuntu" 2022-02-06T19:24:38.386 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-06T19:24:38.387 INFO:teuthology.orchestra.run.smithi084.stdout:ID=ubuntu 2022-02-06T19:24:38.387 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE=debian 2022-02-06T19:24:38.387 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-06T19:24:38.388 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="20.04" 2022-02-06T19:24:38.388 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-06T19:24:38.388 INFO:teuthology.orchestra.run.smithi084.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-06T19:24:38.389 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-06T19:24:38.389 INFO:teuthology.orchestra.run.smithi084.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-06T19:24:38.389 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_CODENAME=focal 2022-02-06T19:24:38.389 INFO:teuthology.orchestra.run.smithi084.stdout:UBUNTU_CODENAME=focal 2022-02-06T19:24:38.390 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2022-02-06T19:24:38.508 INFO:teuthology.dispatcher.supervisor:Running job 6665287 2022-02-06T19:24:38.509 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi/6665287 --name benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} -- /home/teuthworker/archive/benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi/6665287/orig.config.yaml 2022-02-06T19:24:38.513 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi/6665287 2022-02-06T19:24:38.514 INFO:teuthology.dispatcher.supervisor:Job PID: 390 2022-02-06T19:24:38.514 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-07T01:55:07.656 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-07T01:55:07.866 INFO:teuthology.kill:Killing Pids: {390} 2022-02-07T01:55:09.045 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-07T01:55:09.046 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-07T01:55:09.047 INFO:teuthology.misc:Compressing logs... 2022-02-07T01:55:09.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2022-02-07T01:55:09.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2022-02-07T01:55:09.463 INFO:teuthology.misc:Compressing logs... 2022-02-07T01:55:36.848 INFO:teuthology.misc:Compressing logs... 2022-02-07T01:56:49.673 INFO:teuthology.kill:No teuthology processes running 2022-02-07T01:56:50.764 INFO:teuthology.kill:Nuking machines: ['smithi082', 'smithi084'] 2022-02-07T01:56:51.748 INFO:teuthology.kill:2022-02-07 01:56:51,748.748 INFO:teuthology.nuke:targets: 2022-02-07T01:56:51.750 INFO:teuthology.kill: smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDV2y9ZigTuVhIdBXWx3LBFhcEaIYDZA1Vuyx7Wj3ZBhvZbPbmtAXrwFKu51X+OMphWYXqoiwoS/jEMmNm8SmatzRDhwuVlALDnIkMbrcTSgiAxOHOrCOpDSxQiIb0MYVy77i19tlQylwAAswZPbJK+uSXnmRjRKw7Lg4Ma3kn6KZQv5uyJwHxQeqGdc1PXCCwVMCL+oxbrOIayWSHgybP5O4InAZZeTUmib/eHPsFyjE6chgT5oBRtCIPSBX+5sZtspofW9KYvzS975TccOivnuyqbsZmalrMoSFof4iwK/OS4CBSpo9MKqR+utfIg68xjrgk/IQv+J9XXVZd+RNB/DNbfv1xLEGA0ger/L6Kp/JSAWWjyMUH0VgNmLSI0SwHe09WcBc3WRwqmvMKD+SHmAOvS2xIJIhclNZZY1lcfH5lkCmgm4aIMl4upW6H4mxViEDsWf9pXdUcKqwdICXiXhy08lOhVb8dNbai+fwPf9TNvbIu4I8fHWayvfYO4QIM= 2022-02-07T01:56:51.750 INFO:teuthology.kill: smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjWWggjVc2b4rIdOAdYjcR3/ABtKUwoMI98dtWjjBHYPYxjST+6/Mb9qMAdVL2+jvHUXq4C2y01STzU+J/aS0pzuQ0NrUX37jrlhkKfERwZ6EwA3whm76WhWzdak0ZRCpFgpcjIu7AFyxQWCMLvve9xU4+aIbCktyPb3U0c4pW39RMQRTIHwP5y/Mj2IiZwbCHdRwRSHT03wn7nEKsPrakksXQjPih/Obclyu5MPSErJkMLbZdN/Kcao5ubbouIaiXQf56Hj1HzHFsuhCA85TlZJoCYybSAl7bTg1MlTbuPh3NDVOJ7f3Sh+sgM3hYSgy/F7HivrpcR5Np/tr4bDCCzJ1TlHmr/rKVo7+zgUtz68jFQj6mQcw3k1TR8yPa2ZSaYS5wHoaRz/O/F1v5q22R8R8yxVAh/0a+krE7qsIoP6Tg1IJp6jklPReULrqHhuDyEcCckKs/2ZJu465LMq7xb09H/71x87ekANpU8qM3ZPsRZDKQ+25/F2KUSyyjd6M= 2022-02-07T01:56:51.776 INFO:teuthology.kill:2022-02-07 01:56:51,776.776 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T01:56:51.780 INFO:teuthology.kill:2022-02-07 01:56:51,780.780 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T01:56:51.854 INFO:teuthology.kill:2022-02-07 01:56:51,854.854 INFO:teuthology.orchestra.console:Power off smithi084 2022-02-07T01:56:51.865 INFO:teuthology.kill:2022-02-07 01:56:51,865.865 INFO:teuthology.orchestra.console:Power off smithi082 2022-02-07T01:57:00.228 INFO:teuthology.kill:2022-02-07 01:57:00,228.228 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2022-02-07T01:57:00.373 INFO:teuthology.kill:2022-02-07 01:57:00,373.373 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com 2022-02-07T01:57:04.335 INFO:teuthology.kill:2022-02-07 01:57:04,335.335 INFO:teuthology.orchestra.console:Power off for smithi084 completed 2022-02-07T01:57:04.481 INFO:teuthology.kill:2022-02-07 01:57:04,480.480 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com 2022-02-07T01:59:04.782 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-07T01:59:04.892 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi082 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-06_22:53:48-rbd-wip-rbd-quincy-batch-2-distro-default-smithi/6665572 2022-02-07T01:59:04.893 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi084 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-06_22:53:48-rbd-wip-rbd-quincy-batch-2-distro-default-smithi/6665570