2021-12-06T23:25:33.845 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-06T23:25:33.971 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi060.front.sepia.ceph.com' reimaging
2021-12-06T23:25:33.972 INFO:teuthology.lock.ops:Updating [ubuntu@smithi060.front.sepia.ceph.com]: reset os type and version on server
2021-12-06T23:25:33.972 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server
2021-12-06T23:25:33.989 INFO:teuthology.lock.ops:Node 'ubuntu@smithi060.front.sepia.ceph.com' reimaging is complete
2021-12-06T23:25:33.990 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging
2021-12-06T23:25:33.990 INFO:teuthology.lock.ops:Updating [ubuntu@smithi082.front.sepia.ceph.com]: reset os type and version on server
2021-12-06T23:25:33.990 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2021-12-06T23:25:34.008 INFO:teuthology.lock.ops:Node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging is complete
2021-12-06T23:25:34.196 INFO:teuthology.provision.fog.smithi060:Scheduling deploy of centos 8.stream
2021-12-06T23:25:34.216 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of centos 8.stream
2021-12-06T23:25:34.547 INFO:teuthology.orchestra.console:Power off smithi060
2021-12-06T23:25:34.585 INFO:teuthology.orchestra.console:Power off smithi082
2021-12-06T23:25:42.920 INFO:teuthology.orchestra.console:Power off for smithi060 completed
2021-12-06T23:25:42.944 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2021-12-06T23:25:43.022 INFO:teuthology.orchestra.console:Power on smithi060
2021-12-06T23:25:43.046 INFO:teuthology.orchestra.console:Power on smithi082
2021-12-06T23:25:51.409 INFO:teuthology.orchestra.console:Power on for smithi060 completed
2021-12-06T23:25:51.415 INFO:teuthology.orchestra.console:Power on for smithi082 completed
2021-12-06T23:25:51.510 INFO:teuthology.provision.fog.smithi060:Waiting for deploy to finish
2021-12-06T23:25:51.516 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish
2021-12-06T23:30:26.235 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-06T23:30:26.550 INFO:teuthology.provision.fog.smithi060:Node is ready
2021-12-06T23:30:26.568 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060.front.sepia.ceph.com
2021-12-06T23:30:26.628 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60 smithi060.front.sepia.ceph.com smithi060
2021-12-06T23:30:26.973 INFO:teuthology.provision.fog.smithi060:Deploy complete!
2021-12-06T23:30:41.274 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-06T23:30:41.560 INFO:teuthology.provision.fog.smithi082:Node is ready
2021-12-06T23:30:41.577 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082.front.sepia.ceph.com
2021-12-06T23:30:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082
2021-12-06T23:30:41.968 INFO:teuthology.provision.fog.smithi082:Deploy complete!
2021-12-06T23:30:42.147 INFO:teuthology.lock.ops:Checking smithi060.front.sepia.ceph.com
2021-12-06T23:30:42.148 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-06T23:30:42.169 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com
2021-12-06T23:30:42.169 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-06T23:30:42.188 INFO:teuthology.lock.ops:Updating [smithi060.front.sepia.ceph.com]: set os type and version on server
2021-12-06T23:30:42.189 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com'
2021-12-06T23:30:42.444 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64
2021-12-06T23:30:42.502 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream"
2021-12-06T23:30:42.503 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8"
2021-12-06T23:30:42.503 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos"
2021-12-06T23:30:42.503 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora"
2021-12-06T23:30:42.503 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8"
2021-12-06T23:30:42.504 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8"
2021-12-06T23:30:42.504 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-06T23:30:42.504 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31"
2021-12-06T23:30:42.504 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-06T23:30:42.504 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/"
2021-12-06T23:30:42.505 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-06T23:30:42.505 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-06T23:30:42.505 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-06T23:30:42.506 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server
2021-12-06T23:30:42.528 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server
2021-12-06T23:30:42.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2021-12-06T23:30:42.718 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64
2021-12-06T23:30:42.774 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream"
2021-12-06T23:30:42.774 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8"
2021-12-06T23:30:42.775 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos"
2021-12-06T23:30:42.775 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora"
2021-12-06T23:30:42.775 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8"
2021-12-06T23:30:42.775 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8"
2021-12-06T23:30:42.776 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-06T23:30:42.776 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31"
2021-12-06T23:30:42.776 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-06T23:30:42.776 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/"
2021-12-06T23:30:42.777 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-06T23:30:42.777 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-06T23:30:42.777 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-06T23:30:42.778 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2021-12-06T23:30:42.885 INFO:teuthology.dispatcher.supervisor:Running job 6547876
2021-12-06T23:30:42.885 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547876 --name benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547876/orig.config.yaml
2021-12-06T23:30:42.889 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547876
2021-12-06T23:30:42.890 INFO:teuthology.dispatcher.supervisor:Job PID: 29261
2021-12-06T23:30:42.890 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-07T11:31:03.212 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-07T11:31:03.394 INFO:teuthology.kill:Killing Pids: {29261}
2021-12-07T11:31:04.695 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-12-07T11:31:04.695 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-12-07T11:31:04.696 INFO:teuthology.misc:Compressing logs...
2021-12-07T11:31:04.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com'
2021-12-07T11:31:04.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2021-12-07T11:31:05.193 INFO:teuthology.misc:Compressing logs...
2021-12-07T11:31:06.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-07T11:31:06.917 INFO:teuthology.misc:Compressing logs...
2021-12-07T11:31:09.376 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-mds.cephfs.smithi060.lkkyqa.log: file size changed while zipping
2021-12-07T11:31:14.224 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-mon.smithi082.log: file size changed while zipping
2021-12-07T11:31:16.400 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-mds.cephfs.smithi082.ozxygv.log: file size changed while zipping
2021-12-07T11:31:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-mgr.smithi060.ebhcdt.log: file size changed while zipping
2021-12-07T11:31:31.490 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-07T11:31:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-mon.smithi060.log: file size changed while zipping
2021-12-07T11:31:49.557 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-07T11:32:03.543 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-07T11:32:14.387 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-mds.cephfs.smithi082.gqnxmj.log: file size changed while zipping
2021-12-07T11:32:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-07T11:32:27.869 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-07T11:32:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-07T11:32:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/6ae10810-56ed-11ec-8c2e-001a4aab830c/ceph-mds.cephfs.smithi060.onzasl.log: file size changed while zipping
2021-12-07T11:33:38.632 INFO:teuthology.kill:No teuthology processes running
2021-12-07T11:33:39.894 INFO:teuthology.kill:Nuking machines: ['smithi060', 'smithi082']
2021-12-07T11:33:40.939 INFO:teuthology.kill:2021-12-07 11:33:40,939.939 INFO:teuthology.nuke:targets:
2021-12-07T11:33:40.940 INFO:teuthology.kill:  smithi060.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRvyWXiKXRKmvlq01i9zBVi+qPzTUxtUA90bmwgGNpuMNy2nb/80Ly3LER9Bc55AN4/uTjmnni7MvrmV3T36vAEYSR+PC+M+zGD+XMwquQ9WJkFHzyGhw9nt6uqRKka2/jjFJHO47UDoQcM90P7GDn2Sc5/+bamypNxCSGFHTr4ir/iPbLnzfHQslcRNJbZ/iymBp2BzWBjQz+iMhkX0DShojI6bXxn0LpjTYFSccHyiJC7DSliBsgvvviW++8dRtkPofH5lT+aPGaBFRZyvJ5PKQ2pfZr9KiVJE+kFQA0Q2U541QL45kuSAGYEL6+uXKPIBQb+kMF1kAQADx+GISawn1TX5FxKozEKVWbB6xBjzwJFzbLkAKw5GxfLhQKj4e4wxEIKb5l3WolYvldB4L5LEhNvee0a2PgEFxLl7tG7cSL9UNKPL20MRUUJOVBg5eGTyEktHNbVK8QlHFEUIMnM52F6PWFOcTEgAxq07N19+pxfFFnDZY8P4OLXDNk4T8=
2021-12-07T11:33:40.940 INFO:teuthology.kill:  smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQKy0rD+CpnOKweTo3zpE+zls5OVxSVPl8ZhVFvFLCsFYH0vENgSoD5niRR3zDPgvqEUS9NVxrxbacHaz3KJXPvn0mfcwrORzaYWn/NtthPJrwnInqkGw8cdlfuvoB9YptHXQ7Au8aubkcA4RGyl+w0o1G7XEZhb5fvwDEt+UvX3OXiQbvuDNWIbeDg4BIDmgkZxjooMw8SiSE4o6EbKSf9TKZzWF5hsG07cigzkuRHQTm+GRq7tPECRgDYefqehI6tYco1a//+pVMmHOu7Fne9Xmc1g1/0Q53rsyrjGpJzlZ6Ibuu/ZoM35/mfmDG3oYrtYRcoJ/ZvMmy1xRV54EZFDJMdeeE5gP1MR8AvkcxdPyNvF7d/8Uw5Po2wuvSdF5MuypkyVK0uBKMnLgh+0Px2coLOu537lsy6kAxpWSrC1YpLFsFNV9XkoKdxvY1UrKq0wGqsuT+a70l+ifTwP1BzuNMYhnws1kejzqdmoCamR2Pr9sNbm/xf0bmxJqHLBE=
2021-12-07T11:33:40.966 INFO:teuthology.kill:2021-12-07 11:33:40,966.966 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-07T11:33:40.970 INFO:teuthology.kill:2021-12-07 11:33:40,970.970 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-07T11:33:41.049 INFO:teuthology.kill:2021-12-07 11:33:41,049.049 INFO:teuthology.orchestra.console:Power off smithi060
2021-12-07T11:33:41.063 INFO:teuthology.kill:2021-12-07 11:33:41,062.062 INFO:teuthology.orchestra.console:Power off smithi082
2021-12-07T11:33:49.421 INFO:teuthology.kill:2021-12-07 11:33:49,421.421 INFO:teuthology.orchestra.console:Power off for smithi060 completed
2021-12-07T11:33:49.429 INFO:teuthology.kill:2021-12-07 11:33:49,429.429 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2021-12-07T11:33:49.572 INFO:teuthology.kill:2021-12-07 11:33:49,572.572 INFO:teuthology.lock.ops:unlocked: smithi060.front.sepia.ceph.com
2021-12-07T11:33:49.576 INFO:teuthology.kill:2021-12-07 11:33:49,576.576 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com
2021-12-07T11:35:49.906 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-07T11:35:50.022 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi060 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549080
2021-12-07T11:35:50.023 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi082 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549080