2021-12-06T16:44:17.481 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-06T16:44:17.617 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging
2021-12-06T16:44:17.618 INFO:teuthology.lock.ops:Updating [ubuntu@smithi055.front.sepia.ceph.com]: reset os type and version on server
2021-12-06T16:44:17.618 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server
2021-12-06T16:44:17.636 INFO:teuthology.lock.ops:Node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging is complete
2021-12-06T16:44:17.637 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging
2021-12-06T16:44:17.637 INFO:teuthology.lock.ops:Updating [ubuntu@smithi123.front.sepia.ceph.com]: reset os type and version on server
2021-12-06T16:44:17.637 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2021-12-06T16:44:17.657 INFO:teuthology.lock.ops:Node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging is complete
2021-12-06T16:44:18.019 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of centos 8.2
2021-12-06T16:44:18.068 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of centos 8.2
2021-12-06T16:44:19.929 INFO:teuthology.orchestra.console:Power off smithi123
2021-12-06T16:44:19.976 INFO:teuthology.orchestra.console:Power off smithi055
2021-12-06T16:44:28.340 INFO:teuthology.orchestra.console:Power off for smithi055 completed
2021-12-06T16:44:28.441 INFO:teuthology.orchestra.console:Power on smithi055
2021-12-06T16:44:32.427 INFO:teuthology.orchestra.console:Power off for smithi123 completed
2021-12-06T16:44:32.528 INFO:teuthology.orchestra.console:Power on smithi123
2021-12-06T16:44:36.781 INFO:teuthology.orchestra.console:Power on for smithi123 completed
2021-12-06T16:44:36.800 INFO:teuthology.orchestra.console:Power on for smithi055 completed
2021-12-06T16:44:36.883 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish
2021-12-06T16:44:36.901 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish
2021-12-06T16:50:06.189 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-06T16:50:06.517 INFO:teuthology.provision.fog.smithi123:Node is ready
2021-12-06T16:50:06.533 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com
2021-12-06T16:50:06.590 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123
2021-12-06T16:50:06.926 INFO:teuthology.provision.fog.smithi123:Deploy complete!
2021-12-06T16:53:27.120 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-06T16:53:27.440 INFO:teuthology.provision.fog.smithi055:Node is ready
2021-12-06T16:53:27.457 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com
2021-12-06T16:53:27.513 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055
2021-12-06T16:53:27.844 INFO:teuthology.provision.fog.smithi055:Deploy complete!
2021-12-06T16:53:27.996 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com
2021-12-06T16:53:27.997 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-06T16:53:28.017 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com
2021-12-06T16:53:28.018 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-06T16:53:28.036 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server
2021-12-06T16:53:28.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com'
2021-12-06T16:53:28.304 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64
2021-12-06T16:53:28.358 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Linux"
2021-12-06T16:53:28.358 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8 (Core)"
2021-12-06T16:53:28.358 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos"
2021-12-06T16:53:28.359 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora"
2021-12-06T16:53:28.359 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8"
2021-12-06T16:53:28.359 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8"
2021-12-06T16:53:28.359 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-12-06T16:53:28.360 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31"
2021-12-06T16:53:28.360 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-06T16:53:28.360 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.centos.org/"
2021-12-06T16:53:28.360 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-12-06T16:53:28.360 INFO:teuthology.orchestra.run.smithi055.stdout:
2021-12-06T16:53:28.361 INFO:teuthology.orchestra.run.smithi055.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-12-06T16:53:28.361 INFO:teuthology.orchestra.run.smithi055.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-12-06T16:53:28.361 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-12-06T16:53:28.361 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-12-06T16:53:28.362 INFO:teuthology.orchestra.run.smithi055.stdout:
2021-12-06T16:53:28.363 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server
2021-12-06T16:53:28.381 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server
2021-12-06T16:53:28.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2021-12-06T16:53:28.597 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64
2021-12-06T16:53:28.652 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Linux"
2021-12-06T16:53:28.652 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8 (Core)"
2021-12-06T16:53:28.653 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos"
2021-12-06T16:53:28.653 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora"
2021-12-06T16:53:28.653 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8"
2021-12-06T16:53:28.653 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8"
2021-12-06T16:53:28.654 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-12-06T16:53:28.654 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31"
2021-12-06T16:53:28.654 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-06T16:53:28.654 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.centos.org/"
2021-12-06T16:53:28.655 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-12-06T16:53:28.655 INFO:teuthology.orchestra.run.smithi123.stdout:
2021-12-06T16:53:28.655 INFO:teuthology.orchestra.run.smithi123.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-12-06T16:53:28.655 INFO:teuthology.orchestra.run.smithi123.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-12-06T16:53:28.655 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-12-06T16:53:28.656 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-12-06T16:53:28.656 INFO:teuthology.orchestra.run.smithi123.stdout:
2021-12-06T16:53:28.656 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2021-12-06T16:53:28.756 INFO:teuthology.dispatcher.supervisor:Running job 6547726
2021-12-06T16:53:28.757 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547726 --name benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi --description rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} -- /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547726/orig.config.yaml
2021-12-06T16:53:28.762 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547726
2021-12-06T16:53:28.762 INFO:teuthology.dispatcher.supervisor:Job PID: 22644
2021-12-06T16:53:28.763 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-07T04:54:23.781 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-07T04:54:23.964 INFO:teuthology.kill:Killing Pids: {22644}
2021-12-07T04:54:25.126 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a']
2021-12-07T04:54:25.127 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-12-07T04:54:25.128 INFO:teuthology.misc:Compressing logs...
2021-12-07T04:54:25.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com'
2021-12-07T04:54:25.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2021-12-07T04:54:25.726 INFO:teuthology.misc:Compressing logs...
2021-12-07T04:54:26.109 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-07T04:54:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-07T04:54:27.045 INFO:teuthology.misc:Compressing logs...
2021-12-07T04:54:33.746 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-07T04:54:33.945 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-07T04:54:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2021-12-07T04:54:39.350 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-07T04:54:41.125 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-07T04:54:41.654 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2021-12-07T04:54:48.322 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-07T04:54:49.073 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-07T04:54:49.772 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2021-12-07T04:54:54.601 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2021-12-07T04:54:55.080 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-client.rgw.foo.a.log: file size changed while zipping
2021-12-07T04:55:03.091 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/de059c70-56b6-11ec-8c2e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-07T04:56:14.279 INFO:teuthology.orchestra.run.smithi055.stderr:tar: ./de059c70-56b6-11ec-8c2e-001a4aab830c: file changed as we read it
2021-12-07T04:56:14.351 INFO:teuthology.orchestra.run.smithi055.stderr:tar: .: file changed as we read it
2021-12-07T04:57:00.428 INFO:teuthology.kill:No teuthology processes running
2021-12-07T04:57:01.623 INFO:teuthology.kill:Nuking machines: ['smithi055', 'smithi123']
2021-12-07T04:57:02.589 INFO:teuthology.kill:2021-12-07 04:57:02,588.588 INFO:teuthology.nuke:targets:
2021-12-07T04:57:02.590 INFO:teuthology.kill:  smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC39YJdYyrPFmLfSXTeXB5I7Tivwdi+WeqbCYVLbdY/ffN34ENjmtP7MH5uKQ9RobEV8TE2SM9itYeJ3OQCJtweTgEQWyZToNqLdmeOjARmD9ezzTTS6wGTVT8RNr92pDCty7xyO+sZFX/tvnN73EJJokGfd+N44Dg9FZMQ0bRxO+0CHFnwHbbc7IqutqLuUJUwPBcZpjWGCmA2uty2/J+lQ3EHniNRbxjs6Jx6DNdQhi33fsGzgKXq8yRkgrpLHp5BmbHzxNV6DsyNtjb/ymSZbT5mzH0dIAh87uoPAkXEydTq2e6KEq09r1QXJVKO+r163WnRXwQ1kKm5tlZ1B/tL8031iTHd1rqWwTo/cziFe7QYRE6MKtxpAomUpbAPu+Ahxdv+nQ8bT26xNjkEUARDgzaFbx++ZFLKW3pEvUdyoC1u/Zi+wuiHTfzoh+XJbE3mk1APdfGh7Iw7RT3F5MTZg5g2Zmf98ZKCkceYp/JkGd+XpM1BR1jAjWxUxt2Vxfs=
2021-12-07T04:57:02.590 INFO:teuthology.kill:  smithi123.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCBpyD4ri9N4h4vjF0vQoX1sPLABTw2gsCupXFRSc1I9S5kJkQ+uTdC1vOR3ojDfwNCz0FrmyRjOlDch1gS5j22+BMc78Wf+qEfOBP9ZeMqKnz9uM9bj+XO0+j+iRus0vIgyeLWX6C6Cz6cUbxMGKUg6mwUA7L2vfUz5kj780CMEKICKiVugQ5QfFBRDMLESpKYlUr62RLNHa6wM1+R2hLs55P/9gRpOGMxILppmQ9a2/2BEs7fQ5spcNfM7rs2vLvCgoc9IBOMEiSBlVcyWd+89cXxN1zzsXVv2gVxdwylUAtbL6zrRouVyhdPCXGenAGm71msfCJ7D1oySOuuftyqrIx+u/TXaPKIPaTeb/qhjtTKyMxxRpxzTnOfgexZQWrvnVzoQQ7JXAth5WRFWS2yKJWzkYNgi9cZn4mTa1yrk3uXRv3pr4vM4RTyq5U6l/DvIvzKEVSVI268yP59J1NJEM9oI4adcGRRHoqFKYZTWX+AA7Tpa2z82p+93RH45M=
2021-12-07T04:57:02.615 INFO:teuthology.kill:2021-12-07 04:57:02,615.615 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-07T04:57:02.619 INFO:teuthology.kill:2021-12-07 04:57:02,619.619 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-07T04:57:02.708 INFO:teuthology.kill:2021-12-07 04:57:02,707.707 INFO:teuthology.orchestra.console:Power off smithi055
2021-12-07T04:57:02.721 INFO:teuthology.kill:2021-12-07 04:57:02,721.721 INFO:teuthology.orchestra.console:Power off smithi123
2021-12-07T04:57:11.083 INFO:teuthology.kill:2021-12-07 04:57:11,083.083 INFO:teuthology.orchestra.console:Power off for smithi123 completed
2021-12-07T04:57:11.229 INFO:teuthology.kill:2021-12-07 04:57:11,229.229 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com
2021-12-07T04:57:15.189 INFO:teuthology.kill:2021-12-07 04:57:15,189.189 INFO:teuthology.orchestra.console:Power off for smithi055 completed
2021-12-07T04:57:15.340 INFO:teuthology.kill:2021-12-07 04:57:15,340.340 INFO:teuthology.lock.ops:unlocked: smithi055.front.sepia.ceph.com
2021-12-07T04:59:15.659 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-07T04:59:15.755 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi055 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:24:43-upgrade-wip-master_12.6.21-distro-default-smithi/6548796
2021-12-07T04:59:15.756 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi123 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_00:24:43-upgrade-wip-master_12.6.21-distro-default-smithi/6548796