2022-02-07T14:47:33.270 INFO:root:teuthology version: 1.1.0-3094160c
2022-02-07T14:47:33.417 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging
2022-02-07T14:47:33.418 INFO:teuthology.lock.ops:Updating [ubuntu@smithi038.front.sepia.ceph.com]: reset os type and version on server
2022-02-07T14:47:33.418 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-02-07T14:47:33.437 INFO:teuthology.lock.ops:Node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging is complete
2022-02-07T14:47:33.437 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging
2022-02-07T14:47:33.437 INFO:teuthology.lock.ops:Updating [ubuntu@smithi109.front.sepia.ceph.com]: reset os type and version on server
2022-02-07T14:47:33.438 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-02-07T14:47:33.457 INFO:teuthology.lock.ops:Node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging is complete
2022-02-07T14:47:33.658 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 8.stream
2022-02-07T14:47:33.674 INFO:teuthology.provision.fog.smithi109:Scheduling deploy of centos 8.stream
2022-02-07T14:47:34.003 INFO:teuthology.orchestra.console:Power off smithi038
2022-02-07T14:47:34.042 INFO:teuthology.orchestra.console:Power off smithi109
2022-02-07T14:47:46.489 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-02-07T14:47:46.515 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-02-07T14:47:46.593 INFO:teuthology.orchestra.console:Power on smithi038
2022-02-07T14:47:46.623 INFO:teuthology.orchestra.console:Power on smithi109
2022-02-07T14:47:50.873 INFO:teuthology.orchestra.console:Power on for smithi109 completed
2022-02-07T14:47:50.975 INFO:teuthology.provision.fog.smithi109:Waiting for deploy to finish
2022-02-07T14:47:54.962 INFO:teuthology.orchestra.console:Power on for smithi038 completed
2022-02-07T14:47:55.063 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2022-02-07T14:52:12.018 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-07T14:52:12.287 INFO:teuthology.provision.fog.smithi038:Node is ready
2022-02-07T14:52:12.306 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2022-02-07T14:52:12.366 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2022-02-07T14:52:12.734 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2022-02-07T14:52:32.121 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-07T14:52:32.389 INFO:teuthology.provision.fog.smithi109:Node is ready
2022-02-07T14:52:32.406 INFO:teuthology.orchestra.run.smithi109.stdout:smithi109.front.sepia.ceph.com
2022-02-07T14:52:32.464 INFO:teuthology.orchestra.run.smithi109.stdout:172.21.15.109 smithi109.front.sepia.ceph.com smithi109
2022-02-07T14:52:32.822 INFO:teuthology.provision.fog.smithi109:Deploy complete!
2022-02-07T14:52:32.968 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2022-02-07T14:52:32.969 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-07T14:52:32.989 INFO:teuthology.lock.ops:Checking smithi109.front.sepia.ceph.com
2022-02-07T14:52:32.989 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-07T14:52:33.006 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2022-02-07T14:52:33.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-02-07T14:52:33.255 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2022-02-07T14:52:33.313 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2022-02-07T14:52:33.314 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8"
2022-02-07T14:52:33.314 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2022-02-07T14:52:33.314 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2022-02-07T14:52:33.314 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8"
2022-02-07T14:52:33.315 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8"
2022-02-07T14:52:33.315 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-07T14:52:33.315 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2022-02-07T14:52:33.315 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-07T14:52:33.316 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2022-02-07T14:52:33.316 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-07T14:52:33.316 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-07T14:52:33.316 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-07T14:52:33.318 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-02-07T14:52:33.334 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: set os type and version on server
2022-02-07T14:52:33.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-02-07T14:52:33.523 INFO:teuthology.orchestra.run.smithi109.stdout:x86_64
2022-02-07T14:52:33.580 INFO:teuthology.orchestra.run.smithi109.stdout:NAME="CentOS Stream"
2022-02-07T14:52:33.581 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION="8"
2022-02-07T14:52:33.582 INFO:teuthology.orchestra.run.smithi109.stdout:ID="centos"
2022-02-07T14:52:33.582 INFO:teuthology.orchestra.run.smithi109.stdout:ID_LIKE="rhel fedora"
2022-02-07T14:52:33.583 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION_ID="8"
2022-02-07T14:52:33.583 INFO:teuthology.orchestra.run.smithi109.stdout:PLATFORM_ID="platform:el8"
2022-02-07T14:52:33.584 INFO:teuthology.orchestra.run.smithi109.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-07T14:52:33.584 INFO:teuthology.orchestra.run.smithi109.stdout:ANSI_COLOR="0;31"
2022-02-07T14:52:33.585 INFO:teuthology.orchestra.run.smithi109.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-07T14:52:33.585 INFO:teuthology.orchestra.run.smithi109.stdout:HOME_URL="https://centos.org/"
2022-02-07T14:52:33.586 INFO:teuthology.orchestra.run.smithi109.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-07T14:52:33.586 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-07T14:52:33.587 INFO:teuthology.orchestra.run.smithi109.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-07T14:52:33.588 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server
2022-02-07T14:52:33.684 INFO:teuthology.dispatcher.supervisor:Running job 6666470
2022-02-07T14:52:33.685 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-07_14:18:36-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666470 --name benhanokh-2022-02-07_14:18:36-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/benhanokh-2022-02-07_14:18:36-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666470/orig.config.yaml
2022-02-07T14:52:33.688 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-07_14:18:36-rados-WIP_GBH_safe_shutdown_v2A-distro-basic-smithi/6666470
2022-02-07T14:52:33.690 INFO:teuthology.dispatcher.supervisor:Job PID: 20304
2022-02-07T14:52:33.690 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-07T21:23:02.983 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-07T21:23:03.181 INFO:teuthology.kill:Killing Pids: {20304}
2022-02-07T21:23:04.519 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2022-02-07T21:23:04.521 INFO:teuthology.task.internal:roles: ubuntu@smithi109.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2022-02-07T21:23:04.522 INFO:teuthology.misc:Compressing logs...
2022-02-07T21:23:04.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-02-07T21:23:04.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi109.front.sepia.ceph.com'
2022-02-07T21:23:05.060 INFO:teuthology.misc:Compressing logs...
2022-02-07T21:23:05.510 INFO:teuthology.orchestra.run.smithi109.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-07T21:23:05.877 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-07T21:23:09.867 INFO:teuthology.misc:Compressing logs...
2022-02-07T21:23:34.898 INFO:teuthology.kill:No teuthology processes running
2022-02-07T21:23:36.097 INFO:teuthology.kill:Nuking machines: ['smithi038', 'smithi109']
2022-02-07T21:23:37.095 INFO:teuthology.kill:2022-02-07 21:23:37,095.095 INFO:teuthology.nuke:targets:
2022-02-07T21:23:37.096 INFO:teuthology.kill:  smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK5ZLPhQjlC82YMFf/I3Bqv4v38yVKPSjSG3qf35bgB3ov9tnIUZqu1v0ivXUZcTEs6qgZmQSzK38Wxeb0MOdQ9efADcKIirA0CuB9JX04N2+GcHOOdz5Llx8ReDIHmF+iL87Ts6jq0eMfDt/uQ8dkWJqxYljBg1Dd3mQDWf49aD5s0hqXNbonNtfgdzDWtMVe2MfIAfZcrOZQX/tAiwLwHuqasvI129A5vEH4UNcWaYe8lPYpz6M1CwttKjtxUraIthNKVqLRoU9bymiD9O4F/eR7FsddXb/G1RV/pWYeTHoTNYldiBqrM2gSL+5w3wBZhZylPUqqv/1PVrBxAxiyV+djQiwgkpF/UUTolTlTF5uF28rlUP5EH/cMEfM0NjaVp1HaibuaKT6dUYbWQrw0IWDqtopO+DSkQBkMx6aqJ3mTbR2u3dPeIcr/EKFAsaBhzOU3x+Ng8251zIe769OMBNU+kw8XboZXNzOa9k3XbpF9m0sK8bcdeWIdBEx0irc=
2022-02-07T21:23:37.097 INFO:teuthology.kill:  smithi109.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCst5vlDe54ZRJ4ivyDcXVUR6gI2IrApTlLfcS0eqo6LsH2/hIhXzTc39mUl/1oe7ky1+1PZlEfLIu9W6f7dUeikZ64c/FMFEw1IoSnU64XZSdSf7qjBOPq84SjDFE167cjuOlIMeDnE+i4+KxdntXiyM8Jo8GUz0xbUOEzUn7wny7utl+0ehoc3pIPc+gi8PEEqXYUAsennBNE9fP9/Rq2z3X3ugBxB3v1Dm1UXUzXNMWCcaTS2xjrl1s/m5idx5n708rNA+GxRLgWWGiUnNQCURceEXAJfM8sV7cMh9CD+GpAcJw8QLvOXWpJ4IQ0I0UMUi2uWIYYWqUVJFZ6RkVk8zSf6eY4wDeGPlGylfvO4BSD4fnRlE3CoBbCgu7ZXTUIhr2IpQtQ/lwj5EpR7rUXHWmq69vik0PFfnBer1bG0qSQ22GxFZkYKekqmxzjnwCeqRpNkwRoc8ZgxaUyxkuHuB2OgReTKezkyz42X6plKIwvTF2hGVSDfTxyQ0BtcYc=
2022-02-07T21:23:37.124 INFO:teuthology.kill:2022-02-07 21:23:37,124.124 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-07T21:23:37.128 INFO:teuthology.kill:2022-02-07 21:23:37,127.127 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-07T21:23:37.207 INFO:teuthology.kill:2022-02-07 21:23:37,206.206 INFO:teuthology.orchestra.console:Power off smithi109
2022-02-07T21:23:37.220 INFO:teuthology.kill:2022-02-07 21:23:37,220.220 INFO:teuthology.orchestra.console:Power off smithi038
2022-02-07T21:23:49.693 INFO:teuthology.kill:2022-02-07 21:23:49,693.693 INFO:teuthology.orchestra.console:Power off for smithi109 completed
2022-02-07T21:23:49.699 INFO:teuthology.kill:2022-02-07 21:23:49,699.699 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-02-07T21:23:49.846 INFO:teuthology.kill:2022-02-07 21:23:49,845.845 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com
2022-02-07T21:23:49.848 INFO:teuthology.kill:2022-02-07 21:23:49,848.848 INFO:teuthology.lock.ops:unlocked: smithi109.front.sepia.ceph.com
2022-02-07T21:25:50.148 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-07T21:25:50.256 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi109 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-07_20:08:36-rados-wip-yuri4-testing-2022-02-07-1051-octopus-distro-default-smithi/6666826