2022-01-25T06:45:49.494 INFO:root:teuthology version: 1.1.0-7c0cb867
2022-01-25T06:45:49.621 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging
2022-01-25T06:45:49.622 INFO:teuthology.lock.ops:Updating [ubuntu@smithi172.front.sepia.ceph.com]: reset os type and version on server
2022-01-25T06:45:49.622 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2022-01-25T06:45:49.641 INFO:teuthology.lock.ops:Node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging is complete
2022-01-25T06:45:49.642 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging
2022-01-25T06:45:49.643 INFO:teuthology.lock.ops:Updating [ubuntu@smithi176.front.sepia.ceph.com]: reset os type and version on server
2022-01-25T06:45:49.643 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server
2022-01-25T06:45:49.680 INFO:teuthology.lock.ops:Node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging is complete
2022-01-25T06:45:49.866 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 8.stream
2022-01-25T06:45:49.895 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of centos 8.stream
2022-01-25T06:45:50.270 INFO:teuthology.orchestra.console:Power off smithi176
2022-01-25T06:45:50.289 INFO:teuthology.orchestra.console:Power off smithi172
2022-01-25T06:45:58.649 INFO:teuthology.orchestra.console:Power off for smithi176 completed
2022-01-25T06:45:58.750 INFO:teuthology.orchestra.console:Power on smithi176
2022-01-25T06:46:02.768 INFO:teuthology.orchestra.console:Power off for smithi172 completed
2022-01-25T06:46:02.870 INFO:teuthology.orchestra.console:Power on smithi172
2022-01-25T06:46:03.004 INFO:teuthology.orchestra.console:Power on for smithi176 completed
2022-01-25T06:46:03.105 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish
2022-01-25T06:46:07.123 INFO:teuthology.orchestra.console:Power on for smithi172 completed
2022-01-25T06:46:07.225 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish
2022-01-25T06:50:14.370 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-25T06:50:22.027 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2022-01-25T06:50:39.267 INFO:teuthology.provision.fog.smithi176:Node is ready
2022-01-25T06:50:39.292 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com
2022-01-25T06:50:39.352 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176
2022-01-25T06:50:39.707 INFO:teuthology.provision.fog.smithi176:Deploy complete!
2022-01-25T06:50:54.392 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-25T06:50:54.670 INFO:teuthology.provision.fog.smithi172:Node is ready
2022-01-25T06:50:54.686 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172
2022-01-25T06:50:54.743 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172
2022-01-25T06:50:55.089 INFO:teuthology.provision.fog.smithi172:Deploy complete!
2022-01-25T06:50:55.292 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com
2022-01-25T06:50:55.293 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-25T06:50:55.312 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com
2022-01-25T06:50:55.312 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-25T06:50:55.331 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server
2022-01-25T06:50:55.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com'
2022-01-25T06:50:55.519 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64
2022-01-25T06:50:55.575 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream"
2022-01-25T06:50:55.575 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8"
2022-01-25T06:50:55.576 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos"
2022-01-25T06:50:55.576 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora"
2022-01-25T06:50:55.576 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8"
2022-01-25T06:50:55.576 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8"
2022-01-25T06:50:55.576 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-25T06:50:55.577 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31"
2022-01-25T06:50:55.577 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-25T06:50:55.577 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/"
2022-01-25T06:50:55.578 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-25T06:50:55.578 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-25T06:50:55.578 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-25T06:50:55.581 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2022-01-25T06:50:55.601 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server
2022-01-25T06:50:55.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com'
2022-01-25T06:50:55.885 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64
2022-01-25T06:50:55.940 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream"
2022-01-25T06:50:55.941 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="8"
2022-01-25T06:50:55.941 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos"
2022-01-25T06:50:55.941 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora"
2022-01-25T06:50:55.941 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="8"
2022-01-25T06:50:55.942 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el8"
2022-01-25T06:50:55.942 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-25T06:50:55.942 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31"
2022-01-25T06:50:55.943 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-25T06:50:55.943 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/"
2022-01-25T06:50:55.943 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-25T06:50:55.943 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-25T06:50:55.944 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-25T06:50:55.944 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server
2022-01-25T06:50:56.051 INFO:teuthology.dispatcher.supervisor:Running job 6639339
2022-01-25T06:50:56.052 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi/6639339 --name yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi/6639339/orig.config.yaml
2022-01-25T06:50:56.056 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi/6639339
2022-01-25T06:50:56.057 INFO:teuthology.dispatcher.supervisor:Job PID: 20262
2022-01-25T06:50:56.057 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-25T13:21:25.298 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-01-25T13:21:25.501 INFO:teuthology.kill:Killing Pids: {20262}
2022-01-25T13:21:26.703 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2022-01-25T13:21:26.704 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2022-01-25T13:21:26.704 INFO:teuthology.misc:Compressing logs...
2022-01-25T13:21:26.705 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com'
2022-01-25T13:21:26.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com'
2022-01-25T13:21:27.206 INFO:teuthology.misc:Compressing logs...
2022-01-25T13:21:28.324 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-01-25T13:21:28.759 INFO:teuthology.misc:Compressing logs...
2022-01-25T13:21:39.042 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-25T13:21:42.327 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-mon.smithi172.log: file size changed while zipping
2022-01-25T13:21:48.618 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-mgr.smithi172.sjcrtm.log: file size changed while zipping
2022-01-25T13:22:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-25T13:22:03.574 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi176.cyzrxt.log: file size changed while zipping
2022-01-25T13:22:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-25T13:22:13.712 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-25T13:22:18.429 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-25T13:22:20.927 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-25T13:22:23.237 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi176.hpbznp.log: file size changed while zipping
2022-01-25T13:22:27.222 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-mon.smithi176.log: file size changed while zipping
2022-01-25T13:22:38.057 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/9fb88e42-7dac-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi172.gljzbg.log: file size changed while zipping
2022-01-25T13:22:53.770 INFO:teuthology.orchestra.run.smithi172.stderr:tar: ./9fb88e42-7dac-11ec-8c35-001a4aab830c: file changed as we read it
2022-01-25T13:22:53.811 INFO:teuthology.orchestra.run.smithi172.stderr:tar: .: file changed as we read it
2022-01-25T13:23:05.778 INFO:teuthology.kill:No teuthology processes running
2022-01-25T13:23:07.129 INFO:teuthology.kill:Nuking machines: ['smithi172', 'smithi176']
2022-01-25T13:23:08.136 INFO:teuthology.kill:2022-01-25 13:23:08,135.135 INFO:teuthology.nuke:targets:
2022-01-25T13:23:08.137 INFO:teuthology.kill:  smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYOYvJNKWNg578sNubpaBMVjscS/9fNKe992A+GRcchusRQrOLig66/bR6WtY3i0XfPQPXo12inRYgmQkV2nce5eLpdN4gk5lW5wPoHRK9v80FSB9h4zuOvq3J2vZMtzMzWy8M5ZqPGmtGQ+njmqM9XV/afUsX2K0IkTgBb7Z69mrktuIjvf3Vem9vzL4A7m2A4u/lHneIfdzrbB+B2dU4YwS9v61R35SB+/RYMcrOTJz3aoW5V4Ay3AAcshcQwGfe56uwBRnl2qPBHr5RK4tK2SvwhhP6FYeiS9XtmA4mC1mgJaBb+qc53dFkSnfWXYP8tYw5d2QfrLQMN9R8QdCXlo6v5KzmLMlt7bB2LQ1TZ54tx8T6eYyFRh5E5wM1+MLoOYdtuwZ/uP0GpG37PXJbFQK9PJhBB+9tjAJkJsdpEPLILWjnTvsl7iwraRhVJLBtpawgepi29PMopJw8b14K9gy2O1c9EJjOzRXztUgHennRDviZdYKxTaptZZXw7WU=
2022-01-25T13:23:08.137 INFO:teuthology.kill:  smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRcJzTP4HdZFy6fBNrcZQ0vIijiHqb5EGFb7p9C/HxL2VKP0SUHuuGcdj/B8e+N4VARY/MuDmUb25GL31hm4X6JEJfd7Yy37WztlWIKztBgtSN0Gs/QFKCzXh0Qr/56kGjAznSNmFMFt4k7y8WomAiBd7pxeJa3HRy0nCMSyhO8sfE5D7Xb4t2FsuXXwiguxNcJgS2ZAKxBanX0Dy20GxkdkOuU2lzjdm1H0x1MoywRhzXTwHiTgNoz3Q7C5EZWkyH74nxK36ot7ajRngN51c9oeVP8xA9ByJ6v6UqfkZqFeY28UNZNXCkgLCAFGJfGmMuN9suABTOeQwQYXYDc7dgeWO6vEz72guZm5f8xaZq9D6r7Js6t1ky9bfUcx1DhuDPExMFbW2/uFIVfI3kGZw6ZJpR+WsOVV2mF7FH+keYf6NvrSlIBr4ZuNGcwHLAbuvxsk3unTpTpjvlERxea9X07imXa9P1q3Q43kI04tPVY0jCVm5tAdqSmrV14CQLjYk=
2022-01-25T13:23:08.163 INFO:teuthology.kill:2022-01-25 13:23:08,163.163 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-25T13:23:08.167 INFO:teuthology.kill:2022-01-25 13:23:08,167.167 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-25T13:23:08.245 INFO:teuthology.kill:2022-01-25 13:23:08,244.244 INFO:teuthology.orchestra.console:Power off smithi176
2022-01-25T13:23:08.258 INFO:teuthology.kill:2022-01-25 13:23:08,257.257 INFO:teuthology.orchestra.console:Power off smithi172
2022-01-25T13:23:16.617 INFO:teuthology.kill:2022-01-25 13:23:16,616.616 INFO:teuthology.orchestra.console:Power off for smithi176 completed
2022-01-25T13:23:16.622 INFO:teuthology.kill:2022-01-25 13:23:16,622.622 INFO:teuthology.orchestra.console:Power off for smithi172 completed
2022-01-25T13:23:16.762 INFO:teuthology.kill:2022-01-25 13:23:16,761.761 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com
2022-01-25T13:23:16.768 INFO:teuthology.kill:2022-01-25 13:23:16,767.767 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com
2022-01-25T13:25:17.111 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-25T13:25:17.236 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi172 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-25_04:17:02-fs-pacific-distro-default-smithi/6639664
2022-01-25T13:25:17.236 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi176 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-25_04:17:02-fs-pacific-distro-default-smithi/6639664