2022-02-18T17:29:42.454 INFO:root:teuthology version: 1.1.0-eea25212 2022-02-18T17:29:42.580 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi079.front.sepia.ceph.com' reimaging 2022-02-18T17:29:42.581 INFO:teuthology.lock.ops:Updating [ubuntu@smithi079.front.sepia.ceph.com]: reset os type and version on server 2022-02-18T17:29:42.582 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2022-02-18T17:29:42.600 INFO:teuthology.lock.ops:Node 'ubuntu@smithi079.front.sepia.ceph.com' reimaging is complete 2022-02-18T17:29:42.601 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging 2022-02-18T17:29:42.601 INFO:teuthology.lock.ops:Updating [ubuntu@smithi092.front.sepia.ceph.com]: reset os type and version on server 2022-02-18T17:29:42.601 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2022-02-18T17:29:42.620 INFO:teuthology.lock.ops:Node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging is complete 2022-02-18T17:29:42.814 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 8.stream 2022-02-18T17:29:42.835 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 8.stream 2022-02-18T17:29:43.130 INFO:teuthology.orchestra.console:Power off smithi079 2022-02-18T17:29:43.144 INFO:teuthology.orchestra.console:Power off smithi092 2022-02-18T17:29:51.504 INFO:teuthology.orchestra.console:Power off for smithi079 completed 2022-02-18T17:29:51.505 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2022-02-18T17:29:51.606 INFO:teuthology.orchestra.console:Power on smithi079 2022-02-18T17:29:51.614 INFO:teuthology.orchestra.console:Power on smithi092 2022-02-18T17:29:59.971 INFO:teuthology.orchestra.console:Power on for smithi079 completed 2022-02-18T17:29:59.978 INFO:teuthology.orchestra.console:Power on for smithi092 completed 2022-02-18T17:30:00.074 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish 2022-02-18T17:30:00.078 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish 2022-02-18T17:32:31.251 ERROR:teuthology.orchestra.connection:Error authenticating with smithi079.front.sepia.ceph.com: Authentication failed. 2022-02-18T17:34:12.118 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-18T17:34:37.656 INFO:teuthology.provision.fog.smithi079:Node is ready 2022-02-18T17:34:37.679 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com 2022-02-18T17:34:37.738 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079 2022-02-18T17:34:38.105 INFO:teuthology.provision.fog.smithi079:Deploy complete! 2022-02-18T17:34:38.113 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-18T17:34:38.351 INFO:teuthology.provision.fog.smithi092:Node is ready 2022-02-18T17:34:38.370 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com 2022-02-18T17:34:38.430 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092 2022-02-18T17:34:38.797 INFO:teuthology.provision.fog.smithi092:Deploy complete! 2022-02-18T17:34:38.943 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com 2022-02-18T17:34:38.944 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-18T17:34:39.029 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com 2022-02-18T17:34:39.029 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-18T17:34:39.050 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server 2022-02-18T17:34:39.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2022-02-18T17:34:39.248 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2022-02-18T17:34:39.306 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2022-02-18T17:34:39.306 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8" 2022-02-18T17:34:39.306 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2022-02-18T17:34:39.306 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2022-02-18T17:34:39.307 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8" 2022-02-18T17:34:39.307 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2022-02-18T17:34:39.307 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-18T17:34:39.307 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2022-02-18T17:34:39.308 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-18T17:34:39.308 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2022-02-18T17:34:39.308 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-18T17:34:39.308 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-18T17:34:39.308 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-18T17:34:39.310 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2022-02-18T17:34:39.329 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server 2022-02-18T17:34:39.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2022-02-18T17:34:39.523 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2022-02-18T17:34:39.580 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2022-02-18T17:34:39.580 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8" 2022-02-18T17:34:39.581 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2022-02-18T17:34:39.581 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2022-02-18T17:34:39.581 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8" 2022-02-18T17:34:39.581 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2022-02-18T17:34:39.582 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-18T17:34:39.582 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2022-02-18T17:34:39.582 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-18T17:34:39.582 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2022-02-18T17:34:39.582 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-18T17:34:39.583 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-18T17:34:39.583 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-18T17:34:39.583 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2022-02-18T17:34:39.682 INFO:teuthology.dispatcher.supervisor:Running job 6694223 2022-02-18T17:34:39.683 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6694223 --name teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi --description upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} -- /home/teuthworker/archive/teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6694223/orig.config.yaml 2022-02-18T17:34:39.686 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-18_14:22:03-upgrade:octopus-x-quincy-distro-default-smithi/6694223 2022-02-18T17:34:39.687 INFO:teuthology.dispatcher.supervisor:Job PID: 18362 2022-02-18T17:34:39.687 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-19T00:04:50.141 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-19T00:04:50.581 INFO:teuthology.kill:Killing Pids: {18362} 2022-02-19T00:04:51.878 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2022-02-19T00:04:51.879 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2022-02-19T00:04:51.879 INFO:teuthology.misc:Compressing logs... 2022-02-19T00:04:51.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2022-02-19T00:04:52.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2022-02-19T00:04:52.402 INFO:teuthology.misc:Compressing logs... 2022-02-19T00:04:52.792 INFO:teuthology.misc:Compressing logs... 2022-02-19T00:05:01.501 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-02-19T00:05:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-02-19T00:05:06.661 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-mgr.y.log: file size changed while zipping 2022-02-19T00:05:10.769 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-02-19T00:05:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-02-19T00:05:18.949 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-02-19T00:05:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-02-19T00:05:28.074 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-02-19T00:05:32.942 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-02-19T00:05:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-02-19T00:05:42.495 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-02-19T00:05:52.402 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/8d2c367c-90e2-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-02-19T00:06:13.528 INFO:teuthology.kill:No teuthology processes running 2022-02-19T00:06:14.608 INFO:teuthology.kill:Nuking machines: ['smithi079', 'smithi092'] 2022-02-19T00:06:15.569 INFO:teuthology.kill:2022-02-19 00:06:15,568.568 INFO:teuthology.nuke:targets: 2022-02-19T00:06:15.569 INFO:teuthology.kill: smithi079.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIHF+ONas2gO2foff5pPrMlSY5f935oGlbblgmzlQirKd1qVEWe5YhCVxjSrsK6CSHUFdq10SJXRp0XuKPEsdcP0pSjBBUxGhQ/ewpJsLe6PbkaKOt+tvlvPEHs3uAn6uozhJclaDvPc1YHNuXDCdxiNHqI3T5C+u1PtoG3ESR1ID1+sf1WUT2Jxo32dJZkmtXCuVcS7/I5RJD8ci/JUcEMGcFHSe1UWf74HUw+RRB0FiU/D6aXhxoRP5/xR+2/2gb1YzYT1uqRFibmIjThbRaj54L4/w4M+22IN8T3eaayAlxmLN9D/TM+mWogQr6gUrbfXQI/4VO/P34fr9gg/Y1Vn5/jdN8VUpP7u7n3WkYHEtEuUj36aiXr58IbI0nRPnpO02mzedl7ju0uWULD8xxVSEnU4XhppT4WGHSZQaQnQWm/lUKIhooNxw1k2LAHuLnBdCQVY3JajkCCNmV+VAe6eZ3xhUYOy6pamMug2pbXjo2JIbm1/+QN1sXOWqyumk= 2022-02-19T00:06:15.570 INFO:teuthology.kill: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwJ5Uy4Sn3dFxScqFFQGRtKpZ9nS7d89JLRHwmaocNLV+ptmkzrX5uq+mPOYdiXJP9PlsSqDDLpn/6r3885wNqTz1LVaDD+iXHYUFvdDVEzRq9YfsARk3ntdI3cWGhXbmSu5wduvYLReF0CE6sqFO1WMRmzHlsVdZJU3O3vFvCq2fT0sOEVrMb8YvWudjSj8xQt5aLNOHbALqBkL7SGqw8gOwBMqQZy59KGU5K2xYbT/3/bYelbk4RNFN6mFx0JtYGK53Rnxlg2HYnerWtvT5kAyKbBgBbkWFfhgIAFcUT4+ThG+ZX0lbVDAUGOj9XdZk29Bp3Di+KkhVJpfM6Rr68OPRZ6iyuAes1aNfJS4+rfAIaEGHueG9XpiJSSoQt++S4iWB2z06NaZal6zFGnEA2RWsv2iu2YH5TxCR0o6PytP78wKXZIWPH40Y4iMcX8hdY+Gbl5rndEtZJW5TQzCs+jQ9xFjFBX9dEpiLh7NDp3XjBtaRNGSLI9S0NFq5F2S0= 2022-02-19T00:06:15.594 INFO:teuthology.kill:2022-02-19 00:06:15,594.594 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-19T00:06:15.598 INFO:teuthology.kill:2022-02-19 00:06:15,597.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-19T00:06:15.673 INFO:teuthology.kill:2022-02-19 00:06:15,672.672 INFO:teuthology.orchestra.console:Power off smithi092 2022-02-19T00:06:15.685 INFO:teuthology.kill:2022-02-19 00:06:15,684.684 INFO:teuthology.orchestra.console:Power off smithi079 2022-02-19T00:06:24.041 INFO:teuthology.kill:2022-02-19 00:06:24,041.041 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2022-02-19T00:06:24.047 INFO:teuthology.kill:2022-02-19 00:06:24,046.046 INFO:teuthology.orchestra.console:Power off for smithi079 completed 2022-02-19T00:06:24.186 INFO:teuthology.kill:2022-02-19 00:06:24,185.185 INFO:teuthology.lock.ops:unlocked: smithi092.front.sepia.ceph.com 2022-02-19T00:06:24.189 INFO:teuthology.kill:2022-02-19 00:06:24,189.189 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com 2022-02-19T00:08:24.390 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15