2023-05-21T19:14:32.869 INFO:root:teuthology version: 0.0.1.dev100+g01abd1e 2023-05-21T19:14:33.110 INFO:teuthology.lock.ops:Start node 'smithi135.front.sepia.ceph.com' reimaging 2023-05-21T19:14:33.111 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: reset os type and version on server 2023-05-21T19:14:33.112 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2023-05-21T19:14:33.208 INFO:teuthology.lock.ops:Node 'smithi135.front.sepia.ceph.com' reimaging is complete 2023-05-21T19:14:33.208 INFO:teuthology.lock.ops:Start node 'smithi186.front.sepia.ceph.com' reimaging 2023-05-21T19:14:33.208 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: reset os type and version on server 2023-05-21T19:14:33.209 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-05-21T19:14:33.237 INFO:teuthology.lock.ops:Node 'smithi186.front.sepia.ceph.com' reimaging is complete 2023-05-21T19:14:33.377 INFO:teuthology.provision.fog.smithi135:Scheduling deploy of ubuntu 20.04 2023-05-21T19:14:33.401 INFO:teuthology.provision.fog.smithi186:Scheduling deploy of ubuntu 20.04 2023-05-21T19:14:33.609 INFO:teuthology.orchestra.console:Power off smithi135 2023-05-21T19:14:33.646 INFO:teuthology.orchestra.console:Power off smithi186 2023-05-21T19:14:33.915 INFO:teuthology.orchestra.console:Power off for smithi135 completed 2023-05-21T19:14:33.924 INFO:teuthology.orchestra.console:Power off for smithi186 completed 2023-05-21T19:14:34.017 INFO:teuthology.orchestra.console:Power on smithi135 2023-05-21T19:14:34.027 INFO:teuthology.orchestra.console:Power on smithi186 2023-05-21T19:14:38.280 INFO:teuthology.orchestra.console:Power on for smithi135 completed 2023-05-21T19:14:38.384 INFO:teuthology.provision.fog.smithi135:Waiting for deploy to finish 2023-05-21T19:14:42.402 INFO:teuthology.orchestra.console:Power on for smithi186 completed 2023-05-21T19:14:42.503 INFO:teuthology.provision.fog.smithi186:Waiting for deploy to finish 2023-05-21T19:18:31.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-05-21T19:18:40.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-05-21T19:18:42.512 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.135 2023-05-21T19:18:49.744 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-05-21T19:18:51.568 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.135 2023-05-21T19:18:58.800 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-05-21T19:19:00.624 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.135 2023-05-21T19:19:07.856 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-05-21T19:19:09.680 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.135 2023-05-21T19:19:16.912 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-05-21T19:19:18.736 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.135 2023-05-21T19:19:27.792 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.135 2023-05-21T19:20:23.014 WARNING:teuthology.provision.fog:timed out 2023-05-21T19:20:29.294 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-05-21T19:20:29.795 INFO:teuthology.provision.fog.smithi186:Node is ready 2023-05-21T19:20:29.802 INFO:teuthology.orchestra.run.smithi186.stdout:smithi186.front.sepia.ceph.com 2023-05-21T19:20:29.855 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.15.186 smithi186.front.sepia.ceph.com smithi186 2023-05-21T19:20:30.029 INFO:teuthology.provision.fog.smithi186:Deploy complete! 2023-05-21T19:20:33.805 WARNING:teuthology.provision.fog:timed out 2023-05-21T19:20:40.035 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-05-21T19:20:40.516 INFO:teuthology.provision.fog.smithi135:Node is ready 2023-05-21T19:20:40.522 INFO:teuthology.orchestra.run.smithi135.stdout:smithi135.front.sepia.ceph.com 2023-05-21T19:20:40.529 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135 smithi135.front.sepia.ceph.com smithi135 2023-05-21T19:20:40.719 INFO:teuthology.provision.fog.smithi135:Deploy complete! 2023-05-21T19:20:41.118 INFO:teuthology.lock.ops:Checking smithi135.front.sepia.ceph.com 2023-05-21T19:20:41.119 INFO:teuthology.lock.ops:New key found. Updating... 2023-05-21T19:20:41.147 INFO:teuthology.lock.ops:Checking smithi186.front.sepia.ceph.com 2023-05-21T19:20:41.147 INFO:teuthology.lock.ops:New key found. Updating... 2023-05-21T19:20:41.165 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: set os type and version on server 2023-05-21T19:20:41.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-05-21T19:20:41.339 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2023-05-21T19:20:41.346 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Ubuntu" 2023-05-21T19:20:41.346 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-05-21T19:20:41.346 INFO:teuthology.orchestra.run.smithi135.stdout:ID=ubuntu 2023-05-21T19:20:41.346 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE=debian 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="20.04" 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.ubuntu.com/" 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_CODENAME=focal 2023-05-21T19:20:41.347 INFO:teuthology.orchestra.run.smithi135.stdout:UBUNTU_CODENAME=focal 2023-05-21T19:20:41.348 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2023-05-21T19:20:41.372 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: set os type and version on server 2023-05-21T19:20:41.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-05-21T19:20:41.546 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-05-21T19:20:41.552 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Ubuntu" 2023-05-21T19:20:41.553 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-05-21T19:20:41.553 INFO:teuthology.orchestra.run.smithi186.stdout:ID=ubuntu 2023-05-21T19:20:41.553 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE=debian 2023-05-21T19:20:41.553 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-05-21T19:20:41.553 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="20.04" 2023-05-21T19:20:41.553 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.ubuntu.com/" 2023-05-21T19:20:41.554 INFO:teuthology.orchestra.run.smithi186.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-05-21T19:20:41.554 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-05-21T19:20:41.554 INFO:teuthology.orchestra.run.smithi186.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-05-21T19:20:41.554 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_CODENAME=focal 2023-05-21T19:20:41.554 INFO:teuthology.orchestra.run.smithi186.stdout:UBUNTU_CODENAME=focal 2023-05-21T19:20:41.554 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-05-21T19:20:41.705 INFO:teuthology.dispatcher.supervisor:Running job 7280934 2023-05-21T19:20:41.705 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01abd1ed90624b09fd81d9cb3a79e2834351f0ca/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-05-20_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7280934 --name teuthology-2023-05-20_14:23:11-upgrade:quincy-x-reef-distro-default-smithi --description upgrade:quincy-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} -- /home/teuthworker/archive/teuthology-2023-05-20_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7280934/orig.config.yaml 2023-05-21T19:20:41.709 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-05-20_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7280934 2023-05-21T19:20:41.710 INFO:teuthology.dispatcher.supervisor:Job PID: 26591 2023-05-21T19:20:41.710 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-05-22T07:22:41.743 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-05-22T07:22:48.665 INFO:teuthology.kill:Killing Pids: {26591} 2023-05-22T07:22:51.072 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2023-05-22T07:22:51.094 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2023-05-22T07:22:51.106 INFO:teuthology.misc:Compressing logs... 2023-05-22T07:22:51.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-05-22T07:22:55.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-05-22T07:22:55.263 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-22T07:22:55.275 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-22T07:22:55.275 INFO:teuthology.orchestra.run.smithi135.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 50.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-05-22T07:22:55.276 INFO:teuthology.orchestra.run.smithi135.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 88.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-05-22T07:22:55.506 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-22T07:22:55.507 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-22T07:22:55.507 INFO:teuthology.orchestra.run.smithi186.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 50.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-05-22T07:22:55.518 INFO:teuthology.orchestra.run.smithi186.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 88.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-05-22T07:22:56.139 INFO:teuthology.kill:Killing Pids: {26591} 2023-05-22T07:22:58.422 INFO:teuthology.kill:Nuking machines: ['smithi135', 'smithi186'] 2023-05-22T07:22:59.073 INFO:teuthology.kill:2023-05-22 07:22:59,072.072 INFO:teuthology.nuke:targets: 2023-05-22T07:22:59.077 INFO:teuthology.kill: smithi135.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= 2023-05-22T07:22:59.077 INFO:teuthology.kill: smithi186.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= 2023-05-22T07:22:59.078 INFO:teuthology.kill:2023-05-22 07:22:59,072.072 INFO:teuthology.nuke:Checking targets against current locks 2023-05-22T07:22:59.128 INFO:teuthology.kill:2023-05-22 07:22:59,127.127 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-22T07:22:59.152 INFO:teuthology.kill:2023-05-22 07:22:59,152.152 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-22T07:22:59.212 INFO:teuthology.kill:2023-05-22 07:22:59,211.211 INFO:teuthology.orchestra.console:Power off smithi135 2023-05-22T07:22:59.251 INFO:teuthology.kill:2023-05-22 07:22:59,251.251 INFO:teuthology.orchestra.console:Power off smithi186 2023-05-22T07:23:07.591 INFO:teuthology.kill:2023-05-22 07:23:07,591.591 INFO:teuthology.orchestra.console:Power off for smithi135 completed 2023-05-22T07:23:07.748 INFO:teuthology.kill:2023-05-22 07:23:07,748.748 INFO:teuthology.lock.ops:unlocked: smithi135.front.sepia.ceph.com 2023-05-22T07:23:11.734 INFO:teuthology.kill:2023-05-22 07:23:11,734.734 INFO:teuthology.orchestra.console:Power off for smithi186 completed 2023-05-22T07:23:11.888 INFO:teuthology.kill:2023-05-22 07:23:11,888.888 INFO:teuthology.lock.ops:unlocked: smithi186.front.sepia.ceph.com 2023-05-22T07:25:12.360 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-05-22T07:25:12.565 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi135 but it was locked by another job: /home/teuthworker/archive/adking-2023-05-21_23:31:07-orch:cephadm-wip-adk4-testing-2023-05-21-1612-pacific-distro-default-smithi/7282173 2023-05-22T07:25:12.566 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi186 but it was locked by another job: /home/teuthworker/archive/adking-2023-05-21_23:31:07-orch:cephadm-wip-adk4-testing-2023-05-21-1612-pacific-distro-default-smithi/7282173