2021-12-07T00:10:46.855 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-07T00:10:46.986 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging 2021-12-07T00:10:46.987 INFO:teuthology.lock.ops:Updating [ubuntu@smithi023.front.sepia.ceph.com]: reset os type and version on server 2021-12-07T00:10:46.988 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2021-12-07T00:10:47.006 INFO:teuthology.lock.ops:Node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging is complete 2021-12-07T00:10:47.006 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi049.front.sepia.ceph.com' reimaging 2021-12-07T00:10:47.007 INFO:teuthology.lock.ops:Updating [ubuntu@smithi049.front.sepia.ceph.com]: reset os type and version on server 2021-12-07T00:10:47.007 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2021-12-07T00:10:47.027 INFO:teuthology.lock.ops:Node 'ubuntu@smithi049.front.sepia.ceph.com' reimaging is complete 2021-12-07T00:10:47.207 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of ubuntu 20.04 2021-12-07T00:10:47.241 INFO:teuthology.provision.fog.smithi049:Scheduling deploy of ubuntu 20.04 2021-12-07T00:10:47.520 INFO:teuthology.orchestra.console:Power off smithi023 2021-12-07T00:10:47.560 INFO:teuthology.orchestra.console:Power off smithi049 2021-12-07T00:10:55.922 INFO:teuthology.orchestra.console:Power off for smithi049 completed 2021-12-07T00:10:56.024 INFO:teuthology.orchestra.console:Power on smithi049 2021-12-07T00:11:00.009 INFO:teuthology.orchestra.console:Power off for smithi023 completed 2021-12-07T00:11:00.111 INFO:teuthology.orchestra.console:Power on smithi023 2021-12-07T00:11:04.371 INFO:teuthology.orchestra.console:Power on for smithi023 completed 2021-12-07T00:11:04.398 INFO:teuthology.orchestra.console:Power on for smithi049 completed 2021-12-07T00:11:04.474 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish 2021-12-07T00:11:04.499 INFO:teuthology.provision.fog.smithi049:Waiting for deploy to finish 2021-12-07T00:16:51.298 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-07T00:16:51.831 INFO:teuthology.provision.fog.smithi049:Node is ready 2021-12-07T00:16:51.838 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049.front.sepia.ceph.com 2021-12-07T00:16:51.845 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49 smithi049.front.sepia.ceph.com smithi049 2021-12-07T00:16:51.981 INFO:teuthology.provision.fog.smithi049:Deploy complete! 2021-12-07T00:16:54.283 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-07T00:16:54.826 INFO:teuthology.provision.fog.smithi023:Node is ready 2021-12-07T00:16:54.833 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com 2021-12-07T00:16:54.840 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023 2021-12-07T00:16:55.030 INFO:teuthology.provision.fog.smithi023:Deploy complete! 2021-12-07T00:16:55.228 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com 2021-12-07T00:16:55.229 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-07T00:16:55.252 INFO:teuthology.lock.ops:Checking smithi049.front.sepia.ceph.com 2021-12-07T00:16:55.253 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-07T00:16:55.270 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server 2021-12-07T00:16:55.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2021-12-07T00:16:55.450 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2021-12-07T00:16:55.457 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="Ubuntu" 2021-12-07T00:16:55.457 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-12-07T00:16:55.457 INFO:teuthology.orchestra.run.smithi023.stdout:ID=ubuntu 2021-12-07T00:16:55.458 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE=debian 2021-12-07T00:16:55.458 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-12-07T00:16:55.458 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="20.04" 2021-12-07T00:16:55.458 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.ubuntu.com/" 2021-12-07T00:16:55.459 INFO:teuthology.orchestra.run.smithi023.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-12-07T00:16:55.459 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-12-07T00:16:55.459 INFO:teuthology.orchestra.run.smithi023.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-12-07T00:16:55.459 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_CODENAME=focal 2021-12-07T00:16:55.459 INFO:teuthology.orchestra.run.smithi023.stdout:UBUNTU_CODENAME=focal 2021-12-07T00:16:55.461 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2021-12-07T00:16:55.482 INFO:teuthology.lock.ops:Updating [smithi049.front.sepia.ceph.com]: set os type and version on server 2021-12-07T00:16:55.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2021-12-07T00:16:55.670 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2021-12-07T00:16:55.677 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="Ubuntu" 2021-12-07T00:16:55.677 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-12-07T00:16:55.677 INFO:teuthology.orchestra.run.smithi049.stdout:ID=ubuntu 2021-12-07T00:16:55.678 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE=debian 2021-12-07T00:16:55.678 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-12-07T00:16:55.678 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="20.04" 2021-12-07T00:16:55.678 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://www.ubuntu.com/" 2021-12-07T00:16:55.678 INFO:teuthology.orchestra.run.smithi049.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-12-07T00:16:55.679 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-12-07T00:16:55.679 INFO:teuthology.orchestra.run.smithi049.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-12-07T00:16:55.679 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_CODENAME=focal 2021-12-07T00:16:55.679 INFO:teuthology.orchestra.run.smithi049.stdout:UBUNTU_CODENAME=focal 2021-12-07T00:16:55.680 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2021-12-07T00:16:55.773 INFO:teuthology.dispatcher.supervisor:Running job 6547927 2021-12-07T00:16:55.774 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547927 --name benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-agent-small} -- /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547927/orig.config.yaml 2021-12-07T00:16:55.778 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547927 2021-12-07T00:16:55.779 INFO:teuthology.dispatcher.supervisor:Job PID: 14919 2021-12-07T00:16:55.779 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-07T12:17:50.059 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-07T12:17:50.281 INFO:teuthology.kill:Killing Pids: {14919} 2021-12-07T12:17:51.709 INFO:teuthology.task.internal:roles: ubuntu@smithi023.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-12-07T12:17:51.710 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-12-07T12:17:51.710 INFO:teuthology.misc:Compressing logs... 2021-12-07T12:17:51.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2021-12-07T12:17:51.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2021-12-07T12:17:52.177 INFO:teuthology.misc:Compressing logs... 2021-12-07T12:18:01.084 INFO:teuthology.misc:Compressing logs... 2021-12-07T12:18:40.452 INFO:teuthology.kill:No teuthology processes running 2021-12-07T12:18:41.691 INFO:teuthology.kill:Nuking machines: ['smithi023', 'smithi049'] 2021-12-07T12:18:42.663 INFO:teuthology.kill:2021-12-07 12:18:42,663.663 INFO:teuthology.nuke:targets: 2021-12-07T12:18:42.664 INFO:teuthology.kill: smithi023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9STt/VTXv6lWy3HJ9NlX2ARf15gt6Iml1qmq5jNOwABNUhC6QR2ELBzVyT/5ncEr8txdHRWkf3klJpWrexWGXizlbhuSbcXy9elh8n7fmYehi0TYf9dkfdHbdxDh7uPZ0rTJ/5MCIHoKc6pOYRkLrBQa+i5uapPvOq7ntvk4DAMGmGQL9fF8pjgbFzdThyjNNyr30oBmqYgxP/HJ7AgIu+EowDiqMSlYSx1YErySSV7ZZrMJ+FqRc9nzqaV4dxR4b7aymwxSotxen9Zvyid3vAFsbcYHvwf5QVIiZHCW097RYahUcbJYTRVrYeA1iGf6iSGqdBWfWdGEvzDUbPwKtlHLxO8i6XHSlI4auwRclS0Y3ql0oNJy5+o1QwG29ycRHrOEwCGBbVDVs4SwKvHBBgjhRqjDQuwxzhydDnFybX/Oicxw78MLrXUQfZ+v10JmmwW6aKzcV9ZdvHK1hiXZAKWpY5uh1n3/WZWjdCEgUqe/4b4gTkJzM/H33j0FIzVE= 2021-12-07T12:18:42.664 INFO:teuthology.kill: smithi049.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRN8vpB2sYPjh63zxEjbyJTuPZioGpTQORkqpwW8N9+nP3VJPKfZQ5ao3yvvNKForfNOp7843H18W3+Yw1BZRgQwCgYeB2b6bwfF0x1GkMJbMPqcYEz6/93gag3fzhYoeRw53fS6k5pZqvy4Z3STUV8IEbrifSo5IfRX+L1AfleSQwpiM+rtr6s07jvqjRdWnYT0RTW2ogzsiJdR6PBQuEsf96+6MZOtNNSIRD9DpvavbsyXU0K1NKUZdQOJjA0GDhQqSz2KPBwF0XnSCFigKOixThAGBQASD5F0/Z6nHQGuF0Lr/CT6fdSaXwDlIJHzppRF/Kcl1q2HMYmFV+qXB5+1VHTc3y+p+8lnyJkPLjgOnvWTd3nUVJJRj6dHGRpZf6pV5TI8kNxIMdF3oKl//+EszHp439TeriSF6JnAt8iCdTfxhmQdcqv39BGEolyI09R8UoGx219HEgWmJBkIQs2enebYILVLV9JV8Bu9ws7UsjuhIrIVXiQvsz2zqN+/0= 2021-12-07T12:18:42.689 INFO:teuthology.kill:2021-12-07 12:18:42,689.689 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-07T12:18:42.693 INFO:teuthology.kill:2021-12-07 12:18:42,693.693 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-07T12:18:42.773 INFO:teuthology.kill:2021-12-07 12:18:42,772.772 INFO:teuthology.orchestra.console:Power off smithi049 2021-12-07T12:18:42.786 INFO:teuthology.kill:2021-12-07 12:18:42,786.786 INFO:teuthology.orchestra.console:Power off smithi023 2021-12-07T12:18:51.157 INFO:teuthology.kill:2021-12-07 12:18:51,156.156 INFO:teuthology.orchestra.console:Power off for smithi049 completed 2021-12-07T12:18:51.159 INFO:teuthology.kill:2021-12-07 12:18:51,158.158 INFO:teuthology.orchestra.console:Power off for smithi023 completed 2021-12-07T12:18:51.310 INFO:teuthology.kill:2021-12-07 12:18:51,309.309 INFO:teuthology.lock.ops:unlocked: smithi049.front.sepia.ceph.com 2021-12-07T12:18:51.316 INFO:teuthology.kill:2021-12-07 12:18:51,315.315 INFO:teuthology.lock.ops:unlocked: smithi023.front.sepia.ceph.com 2021-12-07T12:20:51.745 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-07T12:20:51.866 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi023 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549176 2021-12-07T12:20:51.866 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi049 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549176