2022-02-11T14:29:01.382 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-11T14:29:01.502 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging 2022-02-11T14:29:01.503 INFO:teuthology.lock.ops:Updating [ubuntu@smithi059.front.sepia.ceph.com]: reset os type and version on server 2022-02-11T14:29:01.504 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-02-11T14:29:01.521 INFO:teuthology.lock.ops:Node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging is complete 2022-02-11T14:29:01.521 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging 2022-02-11T14:29:01.521 INFO:teuthology.lock.ops:Updating [ubuntu@smithi159.front.sepia.ceph.com]: reset os type and version on server 2022-02-11T14:29:01.522 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-02-11T14:29:01.540 INFO:teuthology.lock.ops:Node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging is complete 2022-02-11T14:29:01.732 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of ubuntu 20.04 2022-02-11T14:29:01.750 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of ubuntu 20.04 2022-02-11T14:29:02.073 INFO:teuthology.orchestra.console:Power off smithi059 2022-02-11T14:29:02.089 INFO:teuthology.orchestra.console:Power off smithi159 2022-02-11T14:29:10.455 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-02-11T14:29:10.556 INFO:teuthology.orchestra.console:Power on smithi159 2022-02-11T14:29:14.561 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-02-11T14:29:14.663 INFO:teuthology.orchestra.console:Power on smithi059 2022-02-11T14:29:18.919 INFO:teuthology.orchestra.console:Power on for smithi059 completed 2022-02-11T14:29:18.923 INFO:teuthology.orchestra.console:Power on for smithi159 completed 2022-02-11T14:29:19.020 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2022-02-11T14:29:19.024 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2022-02-11T14:33:56.955 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-11T14:33:57.436 INFO:teuthology.provision.fog.smithi059:Node is ready 2022-02-11T14:33:57.443 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2022-02-11T14:33:57.491 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2022-02-11T14:33:57.693 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2022-02-11T14:34:28.633 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-11T14:34:29.128 INFO:teuthology.provision.fog.smithi159:Node is ready 2022-02-11T14:34:29.134 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2022-02-11T14:34:29.142 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2022-02-11T14:34:29.335 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2022-02-11T14:34:29.522 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2022-02-11T14:34:29.523 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-11T14:34:29.542 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2022-02-11T14:34:29.543 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-11T14:34:29.560 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2022-02-11T14:34:29.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-02-11T14:34:29.830 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2022-02-11T14:34:29.878 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Ubuntu" 2022-02-11T14:34:29.879 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-11T14:34:29.879 INFO:teuthology.orchestra.run.smithi059.stdout:ID=ubuntu 2022-02-11T14:34:29.879 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE=debian 2022-02-11T14:34:29.879 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-11T14:34:29.880 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="20.04" 2022-02-11T14:34:29.880 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-11T14:34:29.880 INFO:teuthology.orchestra.run.smithi059.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-11T14:34:29.880 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-11T14:34:29.880 INFO:teuthology.orchestra.run.smithi059.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-11T14:34:29.881 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_CODENAME=focal 2022-02-11T14:34:29.881 INFO:teuthology.orchestra.run.smithi059.stdout:UBUNTU_CODENAME=focal 2022-02-11T14:34:29.882 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2022-02-11T14:34:29.905 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2022-02-11T14:34:29.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-02-11T14:34:30.070 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2022-02-11T14:34:30.075 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Ubuntu" 2022-02-11T14:34:30.076 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-11T14:34:30.076 INFO:teuthology.orchestra.run.smithi159.stdout:ID=ubuntu 2022-02-11T14:34:30.076 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE=debian 2022-02-11T14:34:30.076 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-11T14:34:30.077 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="20.04" 2022-02-11T14:34:30.077 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-11T14:34:30.077 INFO:teuthology.orchestra.run.smithi159.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-11T14:34:30.077 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-11T14:34:30.077 INFO:teuthology.orchestra.run.smithi159.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-11T14:34:30.078 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_CODENAME=focal 2022-02-11T14:34:30.078 INFO:teuthology.orchestra.run.smithi159.stdout:UBUNTU_CODENAME=focal 2022-02-11T14:34:30.078 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-02-11T14:34:30.184 INFO:teuthology.dispatcher.supervisor:Running job 6676737 2022-02-11T14:34:30.185 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-11_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi/6676737 --name teuthology-2022-02-11_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi --description upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} -- /home/teuthworker/archive/teuthology-2022-02-11_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi/6676737/orig.config.yaml 2022-02-11T14:34:30.188 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-11_14:22:02-upgrade:octopus-x-quincy-distro-default-smithi/6676737 2022-02-11T14:34:30.188 INFO:teuthology.dispatcher.supervisor:Job PID: 21932 2022-02-11T14:34:30.189 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-11T21:04:58.604 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-11T21:04:58.858 INFO:teuthology.kill:Killing Pids: {21932} 2022-02-11T21:05:00.000 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2022-02-11T21:05:00.001 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2022-02-11T21:05:00.001 INFO:teuthology.misc:Compressing logs... 2022-02-11T21:05:00.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2022-02-11T21:05:00.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-02-11T21:05:00.623 INFO:teuthology.misc:Compressing logs... 2022-02-11T21:05:00.944 INFO:teuthology.misc:Compressing logs... 2022-02-11T21:05:05.111 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-mgr.y.log: file size changed while zipping 2022-02-11T21:05:43.546 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-02-11T21:06:09.334 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-02-11T21:06:14.213 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-02-11T21:06:23.033 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-02-11T21:06:26.826 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-02-11T21:06:27.380 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-02-11T21:07:03.979 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-02-11T21:07:30.866 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-02-11T21:07:55.518 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-02-11T21:08:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-02-11T21:09:18.868 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/d5da919c-8b48-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-02-11T21:10:58.474 INFO:teuthology.kill:No teuthology processes running 2022-02-11T21:10:59.660 INFO:teuthology.kill:Nuking machines: ['smithi059', 'smithi159'] 2022-02-11T21:11:00.623 INFO:teuthology.kill:2022-02-11 21:11:00,623.623 INFO:teuthology.nuke:targets: 2022-02-11T21:11:00.624 INFO:teuthology.kill: smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiPx2pczdVK7jb6n6izCgBAE0stVWCQF7C/ShlkyflEUOgaOAkWTaAX3f6OwI2gjpaQh4yAgUhAEUEgYlc/F9SvoQhbWo2h+OHu6OrsnvZo99t9Mu5aqYVSXCGCI+T2o2W1PQ3wIilO4oJkwghVIbr4OLlIjPW9ML8flMRWBazX0/8O+QkEH91HtdJYZ2qfeGZ6TlUan/hS3T4u524dP8WTSgWmf1igF7b/iQ4haeyg0OuZI/UcFcLyAKSdJOiLpyc98pFaK9jBJW8hBSiXwQMgtG1w2t+wZ0ntSTjxKhpQDKmWevXFNItL59Vq1+cDHGQ7+AMbYDbCBsZ/C0+JC0gFfi2+T9kAzZSOTIMbn75EAm+kPKB0tiw3htGyGLsJAThIXgl2n4NAxfJhhtc2gcGNTFO3La1FsZNnhTxKvsMaYkAUg7YhEAkGSCv7b5qzyBEpyYvl407qdwDeiNRJvG33oz8hNWDy0qnwbPRmXhmBq1GJmMQYinvyuz4fdeeGNk= 2022-02-11T21:11:00.625 INFO:teuthology.kill: smithi159.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFDj0FBJrwsggakf6Mml6J87+0QzCdmhu6sJTIPfKfWqsbeUdTHu8TafbdQjzYQmzbDfW5c+lgL4gqN6lNxG6ym3iT67dBfUSv5DouA40DWXRqFMuSgvwKyhkw0NcO2HYe+3Z3vDYpm7iwiC1y33FtJmFYZ+tkvwNPTPfdDaTdPH/lS0m6Pf82hirqisRZ47m2PCKi7UxiVL8ItRMoCT+AdyslGKa1T/1FcGVmrGQI39GeFP53MC/6iQ41Q769TRgkAC5qInOdbEGO9g4rI+Ne7ve5mfgTc7rCUMaZkTbZ/ihRRzOD8PzWYV81uWHXByN4PdJpTmCEn8XJFC3y+cYoVBb2AWyWkVnmiJwpL6YYXPGiJ6DdAiOBNIVzp6cjsVGkLl1LufXSwIpy7oM/0QO0zi52rxnjegnJJEcY9HDCgXSPKEVwOn53kmoNDpXkkq56nuN7R6B4HwzLABbBKZIGu7LXm7oufhHNCTH9zMW6iVl21XWU8MJDLu92VZ0cW+s= 2022-02-11T21:11:00.651 INFO:teuthology.kill:2022-02-11 21:11:00,651.651 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-11T21:11:00.654 INFO:teuthology.kill:2022-02-11 21:11:00,654.654 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-11T21:11:00.736 INFO:teuthology.kill:2022-02-11 21:11:00,735.735 INFO:teuthology.orchestra.console:Power off smithi159 2022-02-11T21:11:00.750 INFO:teuthology.kill:2022-02-11 21:11:00,750.750 INFO:teuthology.orchestra.console:Power off smithi059 2022-02-11T21:11:09.107 INFO:teuthology.kill:2022-02-11 21:11:09,106.106 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-02-11T21:11:09.254 INFO:teuthology.kill:2022-02-11 21:11:09,254.254 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com 2022-02-11T21:11:13.225 INFO:teuthology.kill:2022-02-11 21:11:13,224.224 INFO:teuthology.orchestra.console:Power off for smithi059 completed 2022-02-11T21:11:13.370 INFO:teuthology.kill:2022-02-11 21:11:13,370.370 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2022-02-11T21:13:13.685 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-11T21:13:13.817 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi059 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-11_19:39:19-upgrade-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677305 2022-02-11T21:13:13.818 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi159 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-11_19:39:19-upgrade-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677305