2022-03-26T12:21:41.552 INFO:root:teuthology version: 1.1.0-70390753
2022-03-26T12:21:41.680 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging
2022-03-26T12:21:41.681 INFO:teuthology.lock.ops:Updating [ubuntu@smithi057.front.sepia.ceph.com]: reset os type and version on server
2022-03-26T12:21:41.682 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server
2022-03-26T12:21:41.700 INFO:teuthology.lock.ops:Node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging is complete
2022-03-26T12:21:41.700 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging
2022-03-26T12:21:41.701 INFO:teuthology.lock.ops:Updating [ubuntu@smithi109.front.sepia.ceph.com]: reset os type and version on server
2022-03-26T12:21:41.701 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-03-26T12:21:41.720 INFO:teuthology.lock.ops:Node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging is complete
2022-03-26T12:21:41.933 INFO:teuthology.provision.fog.smithi057:Scheduling deploy of centos 8.stream
2022-03-26T12:21:41.936 INFO:teuthology.provision.fog.smithi109:Scheduling deploy of centos 8.stream
2022-03-26T12:21:42.265 INFO:teuthology.orchestra.console:Power off smithi057
2022-03-26T12:21:42.302 INFO:teuthology.orchestra.console:Power off smithi109
2022-03-26T12:21:50.664 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-03-26T12:21:50.765 INFO:teuthology.orchestra.console:Power on smithi109
2022-03-26T12:21:54.751 INFO:teuthology.orchestra.console:Power off for smithi057 completed
2022-03-26T12:21:54.853 INFO:teuthology.orchestra.console:Power on smithi057
2022-03-26T12:21:59.109 INFO:teuthology.orchestra.console:Power on for smithi057 completed
2022-03-26T12:21:59.128 INFO:teuthology.orchestra.console:Power on for smithi109 completed
2022-03-26T12:21:59.210 INFO:teuthology.provision.fog.smithi057:Waiting for deploy to finish
2022-03-26T12:21:59.229 INFO:teuthology.provision.fog.smithi109:Waiting for deploy to finish
2022-03-26T12:26:05.901 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-26T12:26:26.205 INFO:teuthology.provision.fog.smithi057:Node is ready
2022-03-26T12:26:26.229 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057.front.sepia.ceph.com
2022-03-26T12:26:26.289 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57 smithi057.front.sepia.ceph.com smithi057
2022-03-26T12:26:26.660 INFO:teuthology.provision.fog.smithi057:Deploy complete!
2022-03-26T12:26:31.275 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-26T12:26:31.517 INFO:teuthology.provision.fog.smithi109:Node is ready
2022-03-26T12:26:31.534 INFO:teuthology.orchestra.run.smithi109.stdout:smithi109.front.sepia.ceph.com
2022-03-26T12:26:31.593 INFO:teuthology.orchestra.run.smithi109.stdout:172.21.15.109 smithi109.front.sepia.ceph.com smithi109
2022-03-26T12:26:31.950 INFO:teuthology.provision.fog.smithi109:Deploy complete!
2022-03-26T12:26:32.110 INFO:teuthology.lock.ops:Checking smithi057.front.sepia.ceph.com
2022-03-26T12:26:32.111 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-26T12:26:32.132 INFO:teuthology.lock.ops:Checking smithi109.front.sepia.ceph.com
2022-03-26T12:26:32.132 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-26T12:26:32.151 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: set os type and version on server
2022-03-26T12:26:32.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com'
2022-03-26T12:26:32.356 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64
2022-03-26T12:26:32.418 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream"
2022-03-26T12:26:32.418 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="8"
2022-03-26T12:26:32.418 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos"
2022-03-26T12:26:32.419 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora"
2022-03-26T12:26:32.419 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="8"
2022-03-26T12:26:32.419 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el8"
2022-03-26T12:26:32.419 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-26T12:26:32.420 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31"
2022-03-26T12:26:32.420 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-26T12:26:32.420 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/"
2022-03-26T12:26:32.420 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-26T12:26:32.421 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-26T12:26:32.421 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-26T12:26:32.422 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server
2022-03-26T12:26:32.440 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: set os type and version on server
2022-03-26T12:26:32.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-03-26T12:26:32.627 INFO:teuthology.orchestra.run.smithi109.stdout:x86_64
2022-03-26T12:26:32.682 INFO:teuthology.orchestra.run.smithi109.stdout:NAME="CentOS Stream"
2022-03-26T12:26:32.683 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION="8"
2022-03-26T12:26:32.683 INFO:teuthology.orchestra.run.smithi109.stdout:ID="centos"
2022-03-26T12:26:32.683 INFO:teuthology.orchestra.run.smithi109.stdout:ID_LIKE="rhel fedora"
2022-03-26T12:26:32.684 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION_ID="8"
2022-03-26T12:26:32.684 INFO:teuthology.orchestra.run.smithi109.stdout:PLATFORM_ID="platform:el8"
2022-03-26T12:26:32.684 INFO:teuthology.orchestra.run.smithi109.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-26T12:26:32.684 INFO:teuthology.orchestra.run.smithi109.stdout:ANSI_COLOR="0;31"
2022-03-26T12:26:32.685 INFO:teuthology.orchestra.run.smithi109.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-26T12:26:32.685 INFO:teuthology.orchestra.run.smithi109.stdout:HOME_URL="https://centos.org/"
2022-03-26T12:26:32.685 INFO:teuthology.orchestra.run.smithi109.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-26T12:26:32.685 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-26T12:26:32.685 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-26T12:26:32.686 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-03-26T12:26:32.781 INFO:teuthology.dispatcher.supervisor:Running job 6760734
2022-03-26T12:26:32.781 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_70390753284281fb41f7552547fa9eb007fa4eef/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-25_14:55:10-rados-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi/6760734 --name yuriw-2022-03-25_14:55:10-rados-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2022-03-25_14:55:10-rados-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi/6760734/orig.config.yaml
2022-03-26T12:26:32.786 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-25_14:55:10-rados-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi/6760734
2022-03-26T12:26:32.787 INFO:teuthology.dispatcher.supervisor:Job PID: 22410
2022-03-26T12:26:32.787 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-26T18:57:01.420 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-26T18:57:01.677 INFO:teuthology.kill:Killing Pids: {22410}
2022-03-26T18:57:02.807 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2022-03-26T18:57:02.808 INFO:teuthology.task.internal:roles: ubuntu@smithi109.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2022-03-26T18:57:02.808 INFO:teuthology.misc:Compressing logs...
2022-03-26T18:57:02.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com'
2022-03-26T18:57:03.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-03-26T18:57:03.310 INFO:teuthology.misc:Compressing logs...
2022-03-26T18:57:03.551 INFO:teuthology.misc:Compressing logs...
2022-03-26T18:57:07.865 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-mgr.smithi057.jlttlo.log: file size changed while zipping
2022-03-26T18:57:16.765 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-03-26T18:57:19.430 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-03-26T18:57:21.753 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-mon.smithi109.log: file size changed while zipping
2022-03-26T18:57:29.593 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-03-26T18:57:36.984 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-03-26T18:57:41.021 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-mon.smithi057.log: file size changed while zipping
2022-03-26T18:57:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-03-26T18:59:14.987 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi109.xgnxpu.log: file size changed while zipping
2022-03-26T18:59:23.714 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-mds.cephfs.smithi109.jnlzqm.log: file size changed while zipping
2022-03-26T18:59:33.555 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /var/log/ceph/91912ca0-ad01-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-03-26T19:00:14.611 INFO:teuthology.orchestra.run.smithi109.stderr:tar: .: file changed as we read it
2022-03-26T19:00:14.691 INFO:teuthology.kill:No teuthology processes running
2022-03-26T19:00:15.815 INFO:teuthology.kill:Nuking machines: ['smithi057', 'smithi109']
2022-03-26T19:00:16.788 INFO:teuthology.kill:2022-03-26 19:00:16,788.788 INFO:teuthology.nuke:targets:
2022-03-26T19:00:16.789 INFO:teuthology.kill:  smithi057.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/bv0li6o5PjNoweZeCMf6c0wyo5d/o5ALjnuNMeHVo0CxsX932tFRRsf0+cX+vX3DLnoU5RERYYtjc5ApYXtjAuUHUm+5sl+aLGI7S/oJ67ndCYnwTkupbAL7m0Z7TsFkacDWjPFJckiPj6dHjLQAb7VJeA4GTYhAia/2dAjHO4HuPBkl6/M6CGovIcFRw9WD4tjkCZp4kXvLvmNjnRVmkzruZCSzhj+uuq2xrZ1/Ms+4xU+WNQxOhuUqitJwY9wGlP+7JHc5+4lixEoLRqDRIzJxiMJKrAx5bPGGOHGAB5DMqY/Srk7rB/T/+vTQkQhXXg3U2aiHwKxmJXLEAs5hDjjm1ggj33lNGoDPaeElyd7aGYn2+s6lxHlgceuB3uNIxYEr1p/vE1sozJnxJKosUBKropbekJEmlebqxEffGjcjWSejJvtW/vyx5PfsMgkEANspUndBYstCg2u4WZfiscg/Xi+aJLJ0gtpL+lerscYcBh0Gcj0yL8s4x0fj/1U=
2022-03-26T19:00:16.789 INFO:teuthology.kill:  smithi109.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk4XlAVOhK7WNxkwxy9rakTsK46X0WG7wB23ZxK58wyIdl+1ZzFDjKLn21b1kfd01XaWSwL9xIW0qFfk1xj5io72Yr7NrJnYdtIo1ij35EYEoXKXEKatndI6aFRPV80W2WFK+3g+vScun8+jeTtUvP2Ct5KEEDKcc64eDEgY/VDWG7v6uETxlogcfY7gYprcgSVC3eOTv8F0kDvvVuD0OWypeogfQZznVQ7qVPB/1rxc9PaxpTAoepaH/DozZ/drG+8G+bCndq3A4eORkDGNtBQfsK8NTZJpbbp4TKlogqLvw0PiTpl/EqwpaPA8+DeFYK2xX3iA3gILVB8/1PiQQYx/1D1os0UaFTz/0hN+JPSKwU91+ojzBdPChWu4GBDYnNH6k6NaqppO11g/VX7KUH5DnWsMeWEJLOn3UCQYH6BbcCf19qXB7tB7f9Jpe+6stF0Xa9sxW1q6L1Vqkl3q2DGxfrpoY3wXNNvHn1Tm+q44qO+3ChNd88FhsFQWxXCdU=
2022-03-26T19:00:16.813 INFO:teuthology.kill:2022-03-26 19:00:16,813.813 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-26T19:00:16.816 INFO:teuthology.kill:2022-03-26 19:00:16,816.816 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-26T19:00:16.889 INFO:teuthology.kill:2022-03-26 19:00:16,889.889 INFO:teuthology.orchestra.console:Power off smithi109
2022-03-26T19:00:16.903 INFO:teuthology.kill:2022-03-26 19:00:16,902.902 INFO:teuthology.orchestra.console:Power off smithi057
2022-03-26T19:00:25.262 INFO:teuthology.kill:2022-03-26 19:00:25,261.261 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-03-26T19:00:25.266 INFO:teuthology.kill:2022-03-26 19:00:25,266.266 INFO:teuthology.orchestra.console:Power off for smithi057 completed
2022-03-26T19:00:25.414 INFO:teuthology.kill:2022-03-26 19:00:25,413.413 INFO:teuthology.lock.ops:unlocked: smithi057.front.sepia.ceph.com
2022-03-26T19:00:25.418 INFO:teuthology.kill:2022-03-26 19:00:25,418.418 INFO:teuthology.lock.ops:unlocked: smithi109.front.sepia.ceph.com
2022-03-26T19:02:25.723 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-26T19:02:25.832 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi057 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757557
2022-03-26T19:02:25.833 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi109 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-24_09:22:49-fs-wip-vshankar-fscrypt-20220324-121321-testing-default-smithi/6757557