2022-02-07T15:11:25.203 INFO:root:teuthology version: 1.1.0-3094160c 2022-02-07T15:11:25.324 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging 2022-02-07T15:11:25.325 INFO:teuthology.lock.ops:Updating [ubuntu@smithi134.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T15:11:25.326 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2022-02-07T15:11:25.344 INFO:teuthology.lock.ops:Node 'ubuntu@smithi134.front.sepia.ceph.com' reimaging is complete 2022-02-07T15:11:25.345 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2022-02-07T15:11:25.345 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2022-02-07T15:11:25.345 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-02-07T15:11:25.364 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2022-02-07T15:11:25.551 INFO:teuthology.provision.fog.smithi134:Scheduling deploy of ubuntu 20.04 2022-02-07T15:11:25.565 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of ubuntu 20.04 2022-02-07T15:11:25.910 INFO:teuthology.orchestra.console:Power off smithi185 2022-02-07T15:11:25.989 INFO:teuthology.orchestra.console:Power off smithi134 2022-02-07T15:11:26.051 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-02-07T15:11:26.131 INFO:teuthology.orchestra.console:Power off for smithi134 completed 2022-02-07T15:11:26.153 INFO:teuthology.orchestra.console:Power on smithi185 2022-02-07T15:11:26.233 INFO:teuthology.orchestra.console:Power on smithi134 2022-02-07T15:11:30.484 INFO:teuthology.orchestra.console:Power on for smithi134 completed 2022-02-07T15:11:30.586 INFO:teuthology.provision.fog.smithi134:Waiting for deploy to finish 2022-02-07T15:11:34.520 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2022-02-07T15:11:34.621 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2022-02-07T15:16:34.627 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T15:16:35.163 INFO:teuthology.provision.fog.smithi134:Node is ready 2022-02-07T15:16:35.170 INFO:teuthology.orchestra.run.smithi134.stdout:smithi134.front.sepia.ceph.com 2022-02-07T15:16:35.225 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.15.134 smithi134.front.sepia.ceph.com smithi134 2022-02-07T15:16:35.391 INFO:teuthology.provision.fog.smithi134:Deploy complete! 2022-02-07T15:16:41.784 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-07T15:16:42.267 INFO:teuthology.provision.fog.smithi185:Node is ready 2022-02-07T15:16:42.274 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2022-02-07T15:16:42.282 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2022-02-07T15:16:42.462 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2022-02-07T15:16:42.650 INFO:teuthology.lock.ops:Checking smithi134.front.sepia.ceph.com 2022-02-07T15:16:42.651 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T15:16:42.670 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2022-02-07T15:16:42.671 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-07T15:16:42.688 INFO:teuthology.lock.ops:Updating [smithi134.front.sepia.ceph.com]: set os type and version on server 2022-02-07T15:16:42.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2022-02-07T15:16:42.870 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2022-02-07T15:16:42.920 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Ubuntu" 2022-02-07T15:16:42.920 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-07T15:16:42.921 INFO:teuthology.orchestra.run.smithi134.stdout:ID=ubuntu 2022-02-07T15:16:42.921 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE=debian 2022-02-07T15:16:42.921 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-07T15:16:42.922 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="20.04" 2022-02-07T15:16:42.922 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-07T15:16:42.922 INFO:teuthology.orchestra.run.smithi134.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-07T15:16:42.922 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-07T15:16:42.923 INFO:teuthology.orchestra.run.smithi134.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-07T15:16:42.923 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_CODENAME=focal 2022-02-07T15:16:42.923 INFO:teuthology.orchestra.run.smithi134.stdout:UBUNTU_CODENAME=focal 2022-02-07T15:16:42.924 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2022-02-07T15:16:42.940 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2022-02-07T15:16:42.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-02-07T15:16:43.115 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2022-02-07T15:16:43.120 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Ubuntu" 2022-02-07T15:16:43.120 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-07T15:16:43.121 INFO:teuthology.orchestra.run.smithi185.stdout:ID=ubuntu 2022-02-07T15:16:43.121 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE=debian 2022-02-07T15:16:43.121 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-07T15:16:43.121 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="20.04" 2022-02-07T15:16:43.121 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-07T15:16:43.122 INFO:teuthology.orchestra.run.smithi185.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-07T15:16:43.122 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-07T15:16:43.122 INFO:teuthology.orchestra.run.smithi185.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-07T15:16:43.122 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_CODENAME=focal 2022-02-07T15:16:43.123 INFO:teuthology.orchestra.run.smithi185.stdout:UBUNTU_CODENAME=focal 2022-02-07T15:16:43.123 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-02-07T15:16:43.221 INFO:teuthology.dispatcher.supervisor:Running job 6666511 2022-02-07T15:16:43.222 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666511 --name benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} -- /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666511/orig.config.yaml 2022-02-07T15:16:43.225 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-07_14:20:29-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666511 2022-02-07T15:16:43.226 INFO:teuthology.dispatcher.supervisor:Job PID: 14231 2022-02-07T15:16:43.226 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-07T21:47:12.031 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-07T21:47:12.189 INFO:teuthology.kill:Killing Pids: {14231} 2022-02-07T21:47:13.404 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-07T21:47:13.405 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-07T21:47:13.405 INFO:teuthology.misc:Compressing logs... 2022-02-07T21:47:13.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2022-02-07T21:47:13.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-02-07T21:47:13.827 INFO:teuthology.misc:Compressing logs... 2022-02-07T21:47:14.689 INFO:teuthology.misc:Compressing logs... 2022-02-07T21:49:18.485 INFO:teuthology.kill:No teuthology processes running 2022-02-07T21:49:19.734 INFO:teuthology.kill:Nuking machines: ['smithi134', 'smithi185'] 2022-02-07T21:49:20.693 INFO:teuthology.kill:2022-02-07 21:49:20,693.693 INFO:teuthology.nuke:targets: 2022-02-07T21:49:20.694 INFO:teuthology.kill: smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5oRSYe53a7qAdFZaqeFAXF7bS7SUXtopp5wUFv1hpabgm26NpbyxUBhjxkHL66eJqJ+S/0HqrFH532Hr+JNtAgiyLIB2HF0K/qweucX95pb3OSG5sZ1EV/Pl5NRKS/lAoUGNv62EIVvz09/QqYNNck00ory+MKHB+UeNgU89pQSpYegTx4z67Sn3Cyop2J3L4vOhMw+BY1pQGyqApaKGHPCEWeFmMy28Vn3Hr30Mo0KyC0XgY2Bs9i1JQZKn+aCyG0sDqsC9X67FDR/zMAk41hE9ubZwS8tjN930mupdZlXY/VqGb47V5Zhhg0AKO1eXtd84mkQ/QifcarCr1kszG/jBYa872adnuKQL0pLofAs3PrEyTUgB482vxawDDnkeG0PgYAHLAYKskd/SzpNrq2akgl1hHfRPg48ywqxeWILu6FQbPonFfy259uR2PxFpiAOqwqmzQnj4CxeMSNUwIYLV6IaXfx7yR88lnGdEK97LyPb4RVzO8IfpajNQnXh8= 2022-02-07T21:49:20.694 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2TUKe66sKwVI3PR/L1ulpjM8MJo1t/KmI/leImsAKBkb6Z3zSSTjIO+dis4Uch7dAr5fv/vnvG+5yJG5x8qC7xrb9JEt/aa7PkUEILZXnJ27l5OErKyKy/+BvsUkxtAYb0VqRpo4iD2TrBxfHUayZl+CqeEI9exGzTCoPvJ6gVmWpYcztuOaQVoFK9h6Bw6034yEdFhQC7iLjsbZgBDm+pUQ4ijxoo3Z+L3rKi7/kQcs+llcppAPQ85e5ItvbhtOfMJcz6IGKRQeZl99seHj+t88uwtIRPCUV5GXG06sXVPrDp1lQ5MfRyv8kRUPMXIeiLpCShTD2Ns6i826rhUZOLjS5SIJxm5KEkLuWttEKSmL5CF+bRv3JmuFYMLXoupjc0QDgNWoRhbhSfnjg1qlaNXMxsmFufGS8shIHSoTq8f7Kz/3FQ7ian8RpJ/58N9apcYUpTqxwHgUXaTwft4OXvytrAJVYBQwDREHQwwIaHPXTxx//YoV4RL0oljIE/a0= 2022-02-07T21:49:20.720 INFO:teuthology.kill:2022-02-07 21:49:20,719.719 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T21:49:20.723 INFO:teuthology.kill:2022-02-07 21:49:20,723.723 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T21:49:20.799 INFO:teuthology.kill:2022-02-07 21:49:20,798.798 INFO:teuthology.orchestra.console:Power off smithi185 2022-02-07T21:49:20.813 INFO:teuthology.kill:2022-02-07 21:49:20,813.813 INFO:teuthology.orchestra.console:Power off smithi134 2022-02-07T21:49:29.173 INFO:teuthology.kill:2022-02-07 21:49:29,173.173 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-02-07T21:49:29.320 INFO:teuthology.kill:2022-02-07 21:49:29,319.319 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com 2022-02-07T21:49:33.288 INFO:teuthology.kill:2022-02-07 21:49:33,287.287 INFO:teuthology.orchestra.console:Power off for smithi134 completed 2022-02-07T21:49:33.433 INFO:teuthology.kill:2022-02-07 21:49:33,433.433 INFO:teuthology.lock.ops:unlocked: smithi134.front.sepia.ceph.com 2022-02-07T21:51:33.753 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-07T21:51:33.859 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi134 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6666941 2022-02-07T21:51:33.860 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6666941