2021-11-04T04:05:15.218 INFO:root:teuthology version: 1.1.0-c56135d1
2021-11-04T04:05:15.346 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi111.front.sepia.ceph.com' reimaging
2021-11-04T04:05:15.347 INFO:teuthology.lock.ops:Updating [ubuntu@smithi111.front.sepia.ceph.com]: reset os type and version on server
2021-11-04T04:05:15.348 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server
2021-11-04T04:05:15.368 INFO:teuthology.lock.ops:Node 'ubuntu@smithi111.front.sepia.ceph.com' reimaging is complete
2021-11-04T04:05:15.369 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging
2021-11-04T04:05:15.369 INFO:teuthology.lock.ops:Updating [ubuntu@smithi112.front.sepia.ceph.com]: reset os type and version on server
2021-11-04T04:05:15.370 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server
2021-11-04T04:05:15.397 INFO:teuthology.lock.ops:Node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging is complete
2021-11-04T04:05:15.581 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of centos 8.3
2021-11-04T04:05:15.624 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 8.3
2021-11-04T04:05:15.922 INFO:teuthology.orchestra.console:Power off smithi111
2021-11-04T04:05:15.971 INFO:teuthology.orchestra.console:Power off smithi112
2021-11-04T04:05:24.307 INFO:teuthology.orchestra.console:Power off for smithi111 completed
2021-11-04T04:05:24.408 INFO:teuthology.orchestra.console:Power on smithi111
2021-11-04T04:05:28.454 INFO:teuthology.orchestra.console:Power off for smithi112 completed
2021-11-04T04:05:28.556 INFO:teuthology.orchestra.console:Power on smithi112
2021-11-04T04:05:32.810 INFO:teuthology.orchestra.console:Power on for smithi112 completed
2021-11-04T04:05:32.903 INFO:teuthology.orchestra.console:Power on for smithi111 completed
2021-11-04T04:05:32.912 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish
2021-11-04T04:05:33.005 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish
2021-11-04T04:08:19.224 ERROR:teuthology.orchestra.connection:Error authenticating with smithi111.front.sepia.ceph.com: Authentication failed.
2021-11-04T04:09:45.059 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-04T04:10:00.853 INFO:teuthology.provision.fog.smithi112:Node is ready
2021-11-04T04:10:00.872 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com
2021-11-04T04:10:00.929 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112
2021-11-04T04:10:01.272 INFO:teuthology.provision.fog.smithi112:Deploy complete!
2021-11-04T04:10:35.570 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-04T04:10:35.868 INFO:teuthology.provision.fog.smithi111:Node is ready
2021-11-04T04:10:35.925 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111.front.sepia.ceph.com
2021-11-04T04:10:35.982 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111 smithi111.front.sepia.ceph.com smithi111
2021-11-04T04:10:36.315 INFO:teuthology.provision.fog.smithi111:Deploy complete!
2021-11-04T04:10:36.507 INFO:teuthology.lock.ops:Checking smithi111.front.sepia.ceph.com
2021-11-04T04:10:36.509 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-04T04:10:36.529 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com
2021-11-04T04:10:36.530 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-04T04:10:36.556 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: set os type and version on server
2021-11-04T04:10:36.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com'
2021-11-04T04:10:36.761 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64
2021-11-04T04:10:36.816 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Linux"
2021-11-04T04:10:36.816 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8"
2021-11-04T04:10:36.816 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos"
2021-11-04T04:10:36.817 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora"
2021-11-04T04:10:36.817 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8"
2021-11-04T04:10:36.817 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8"
2021-11-04T04:10:36.817 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Linux 8"
2021-11-04T04:10:36.818 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31"
2021-11-04T04:10:36.818 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-04T04:10:36.818 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/"
2021-11-04T04:10:36.818 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-11-04T04:10:36.818 INFO:teuthology.orchestra.run.smithi111.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-11-04T04:10:36.819 INFO:teuthology.orchestra.run.smithi111.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-11-04T04:10:36.820 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server
2021-11-04T04:10:36.838 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server
2021-11-04T04:10:36.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com'
2021-11-04T04:10:37.109 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64
2021-11-04T04:10:37.164 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Linux"
2021-11-04T04:10:37.164 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8"
2021-11-04T04:10:37.165 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos"
2021-11-04T04:10:37.165 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora"
2021-11-04T04:10:37.165 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8"
2021-11-04T04:10:37.165 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8"
2021-11-04T04:10:37.166 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Linux 8"
2021-11-04T04:10:37.166 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31"
2021-11-04T04:10:37.166 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-04T04:10:37.166 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/"
2021-11-04T04:10:37.166 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-11-04T04:10:37.167 INFO:teuthology.orchestra.run.smithi112.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-11-04T04:10:37.167 INFO:teuthology.orchestra.run.smithi112.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-11-04T04:10:37.168 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server
2021-11-04T04:10:37.273 INFO:teuthology.dispatcher.supervisor:Running job 6482536
2021-11-04T04:10:37.274 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482536 --name yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482536/orig.config.yaml
2021-11-04T04:10:37.277 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482536
2021-11-04T04:10:37.277 INFO:teuthology.dispatcher.supervisor:Job PID: 14527
2021-11-04T04:10:37.278 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-04T16:10:57.964 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-04T16:10:58.149 INFO:teuthology.kill:Killing Pids: {14527}
2021-11-04T16:10:59.367 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-04T16:10:59.368 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-04T16:10:59.369 INFO:teuthology.misc:Compressing logs...
2021-11-04T16:10:59.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com'
2021-11-04T16:10:59.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com'
2021-11-04T16:10:59.874 INFO:teuthology.misc:Compressing logs...
2021-11-04T16:11:00.190 INFO:teuthology.misc:Compressing logs...
2021-11-04T16:11:10.255 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-mds.cephfs.smithi111.pinyww.log: file size changed while zipping
2021-11-04T16:11:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-11-04T16:11:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-mds.cephfs.smithi112.etxbqp.log: file size changed while zipping
2021-11-04T16:11:50.595 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-mds.cephfs.smithi111.aimkcx.log: file size changed while zipping
2021-11-04T16:12:06.356 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-11-04T16:12:13.092 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-mon.smithi111.log: file size changed while zipping
2021-11-04T16:12:20.657 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-11-04T16:12:27.530 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-mon.smithi112.log: file size changed while zipping
2021-11-04T16:12:27.775 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-mds.cephfs.smithi112.sepqlx.log: file size changed while zipping
2021-11-04T16:12:28.937 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-11-04T16:12:42.730 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-11-04T16:12:53.509 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-mgr.smithi111.nrdnpp.log: file size changed while zipping
2021-11-04T16:13:10.188 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/e34571d2-3d26-11ec-8c28-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-11-04T16:13:54.858 INFO:teuthology.kill:No teuthology processes running
2021-11-04T16:13:56.094 INFO:teuthology.kill:Nuking machines: ['smithi111', 'smithi112']
2021-11-04T16:13:57.087 INFO:teuthology.kill:2021-11-04 16:13:57,087.087 INFO:teuthology.nuke:targets:
2021-11-04T16:13:57.088 INFO:teuthology.kill:  smithi111.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/rZjcY2+GepHmTdkVwgX9qWFx0PV0TNt4m6U1zv7812vSF1abkh47p3/8+niM2SE69oiPxGjyOnwE74z/F2AK7Of5YsBfhJWKssBmUDGTDJ7vBZeQL6egdXANLF7rnA9g4gKzx+l46N11idrgoC7YrHpKpGlRoGk3ztf5v90s+3VwzhnWuxeOP7gnt7U3/NMWNNbufd1bWhT3N2TSkOtTMhOTAQxQywldyFeSZwSXT7YUUn9w8x/N/rWHqNtUdjCuA82+4nOwOi6pIFzrtrB0SKOSXzGzaLHG+tT42PYy6eQcCGwqOpWosxd8ZCcDB3OvNy19pKTeKKuiFT2W+uhWj3A/UdqW5DXYXOW9ssxyFE+nzugT59uUXPadSylYgrNoXTjRR01BLAQ03LLsmIjNpLalcEFEYj5QRh4nGSnJSymLl/l0EON1G/yqdkRLoUQdiOpe2TuTK5Buli3KL2o5UwQ5unItUggQB9ozVkPeASkbOEhEOqqfwQSxyM8Y7Cc=
2021-11-04T16:13:57.089 INFO:teuthology.kill:  smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkQMEFC6drEqJhjuSd0I4Rq7ilZwId7/MA57ZHDPqD234SWYf1qswKmxB+oTn0Gf6MbmFKj4V16ULnJcotQ2HvoDn0VmjeXHEThR/6Oy1LbR8vswcAMu9TCvb340BTZUhJUl+edobrUldGff71rG+0tdj/DeE2caY/nC8M3E42fJ/q8eGjjd9v9nfKFykY6opAlGNIVX6ZsqRILN1HhppfGi/TVOdL8iSrMnLnxdpWHvCF8lnN7vYJLGKmtgPZ02refwIz5QzWanDf8sQYlIBWq+b9Y9p/heBAAbGIHcy8FZgvMGN65OrM3lWcG7YbKqcsX2IQQuXl5OcfubfkwYQ31uttCOiLCUGAiEeVlKUAuVdLwBtv/DtQNsnW24OseCOqzr1qBna2gaO7eIIbFmuqlBR/ZTAag2Obf7qFY6SmOkTQt5eL69TvNoe5KZQinA39cwz0iEzqiRkC+0mJt+6OX5TNdK1HQHI05BrdW7+hbxCz8tvLsS++iQu7mCJRWSs=
2021-11-04T16:13:57.115 INFO:teuthology.kill:2021-11-04 16:13:57,115.115 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-04T16:13:57.119 INFO:teuthology.kill:2021-11-04 16:13:57,119.119 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-04T16:13:57.206 INFO:teuthology.kill:2021-11-04 16:13:57,206.206 INFO:teuthology.orchestra.console:Power off smithi111
2021-11-04T16:13:57.221 INFO:teuthology.kill:2021-11-04 16:13:57,221.221 INFO:teuthology.orchestra.console:Power off smithi112
2021-11-04T16:14:09.695 INFO:teuthology.kill:2021-11-04 16:14:09,695.695 INFO:teuthology.orchestra.console:Power off for smithi111 completed
2021-11-04T16:14:09.700 INFO:teuthology.kill:2021-11-04 16:14:09,700.700 INFO:teuthology.orchestra.console:Power off for smithi112 completed
2021-11-04T16:14:09.840 INFO:teuthology.kill:2021-11-04 16:14:09,839.839 INFO:teuthology.lock.ops:unlocked: smithi111.front.sepia.ceph.com
2021-11-04T16:14:09.847 INFO:teuthology.kill:2021-11-04 16:14:09,846.846 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com
2021-11-04T16:16:10.071 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-04T16:16:10.189 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi111 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-02_19:49:55-rados-wip-yuri8-testing-2021-11-02-1009-pacific-distro-basic-smithi/6479180
2021-11-04T16:16:10.189 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi112 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-02_19:49:55-rados-wip-yuri8-testing-2021-11-02-1009-pacific-distro-basic-smithi/6479180