2023-01-14T00:14:16.471 INFO:root:teuthology version: 0.0.1.dev18+g19d18db
2023-01-14T00:14:16.632 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging
2023-01-14T00:14:16.633 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server
2023-01-14T00:14:16.634 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2023-01-14T00:14:16.651 INFO:teuthology.lock.ops:Node 'smithi119.front.sepia.ceph.com' reimaging is complete
2023-01-14T00:14:16.651 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging
2023-01-14T00:14:16.651 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server
2023-01-14T00:14:16.652 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2023-01-14T00:14:16.668 INFO:teuthology.lock.ops:Node 'smithi167.front.sepia.ceph.com' reimaging is complete
2023-01-14T00:14:16.669 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging
2023-01-14T00:14:16.669 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server
2023-01-14T00:14:16.669 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2023-01-14T00:14:16.684 INFO:teuthology.lock.ops:Node 'smithi178.front.sepia.ceph.com' reimaging is complete
2023-01-14T00:14:16.803 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of rhel 8.6
2023-01-14T00:14:16.818 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of rhel 8.6
2023-01-14T00:14:16.823 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of rhel 8.6
2023-01-14T00:14:17.030 INFO:teuthology.orchestra.console:Power off smithi167
2023-01-14T00:14:17.045 INFO:teuthology.orchestra.console:Power off smithi119
2023-01-14T00:14:17.059 INFO:teuthology.orchestra.console:Power off smithi178
2023-01-14T00:14:25.434 INFO:teuthology.orchestra.console:Power off for smithi178 completed
2023-01-14T00:14:25.535 INFO:teuthology.orchestra.console:Power on smithi178
2023-01-14T00:14:29.534 INFO:teuthology.orchestra.console:Power off for smithi119 completed
2023-01-14T00:14:29.541 INFO:teuthology.orchestra.console:Power off for smithi167 completed
2023-01-14T00:14:29.635 INFO:teuthology.orchestra.console:Power on smithi119
2023-01-14T00:14:29.645 INFO:teuthology.orchestra.console:Power on smithi167
2023-01-14T00:14:33.908 INFO:teuthology.orchestra.console:Power on for smithi178 completed
2023-01-14T00:14:34.009 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish
2023-01-14T00:14:38.009 INFO:teuthology.orchestra.console:Power on for smithi119 completed
2023-01-14T00:14:38.017 INFO:teuthology.orchestra.console:Power on for smithi167 completed
2023-01-14T00:14:38.110 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish
2023-01-14T00:14:38.118 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish
2023-01-14T00:17:22.988 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:17:26.988 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-01-14T00:17:26.988 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:17:32.048 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:17:36.044 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-01-14T00:17:36.044 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:17:41.132 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:17:45.104 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-01-14T00:17:45.105 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:17:50.188 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:17:54.188 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-01-14T00:17:54.189 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:17:59.244 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:18:03.244 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-01-14T00:18:03.248 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:18:08.304 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:18:12.300 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-01-14T00:18:12.300 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:18:17.388 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:18:21.360 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:18:26.448 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-01-14T00:18:45.836 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:18:49.905 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-14T00:18:50.193 INFO:teuthology.provision.fog.smithi119:Node is ready
2023-01-14T00:18:50.212 INFO:teuthology.orchestra.run.smithi119.stdout:smithi022
2023-01-14T00:18:50.269 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-01-14T00:18:50.618 INFO:teuthology.provision.fog.smithi119:Deploy complete!
2023-01-14T00:18:54.896 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-01-14T00:19:01.508 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-14T00:19:01.966 INFO:teuthology.provision.fog.smithi167:Node is ready
2023-01-14T00:19:01.982 INFO:teuthology.orchestra.run.smithi167.stdout:smithi022
2023-01-14T00:19:02.040 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-01-14T00:19:02.484 INFO:teuthology.provision.fog.smithi167:Deploy complete!
2023-01-14T00:19:32.454 WARNING:teuthology.provision.fog:timed out
2023-01-14T00:19:38.806 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-14T00:19:39.092 INFO:teuthology.provision.fog.smithi178:Node is ready
2023-01-14T00:19:39.114 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com
2023-01-14T00:19:39.175 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178
2023-01-14T00:19:39.518 INFO:teuthology.provision.fog.smithi178:Deploy complete!
2023-01-14T00:19:39.621 ERROR:paramiko.transport:Unknown exception: The socket has already been closed by another greenlet
2023-01-14T00:19:39.623 ERROR:paramiko.transport:Traceback (most recent call last):
2023-01-14T00:19:39.623 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run
2023-01-14T00:19:39.623 ERROR:paramiko.transport:    ptype, m = self.packetizer.read_message()
2023-01-14T00:19:39.624 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message
2023-01-14T00:19:39.624 ERROR:paramiko.transport:    header = self.read_all(self.__block_size_in, check_rekey=True)
2023-01-14T00:19:39.624 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all
2023-01-14T00:19:39.625 ERROR:paramiko.transport:    x = self.__socket.recv(n)
2023-01-14T00:19:39.625 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 663, in recv
2023-01-14T00:19:39.625 ERROR:paramiko.transport:    self._wait(self._read_event)
2023-01-14T00:19:39.625 ERROR:paramiko.transport:  File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
2023-01-14T00:19:39.626 ERROR:paramiko.transport:  File "src/gevent/_hub_primitives.py", line 321, in gevent._gevent_c_hub_primitives.wait_on_socket
2023-01-14T00:19:39.626 ERROR:paramiko.transport:gevent.exceptions.ConcurrentObjectUseError: The socket has already been closed by another greenlet
2023-01-14T00:19:39.626 ERROR:paramiko.transport:
2023-01-14T00:19:39.757 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com
2023-01-14T00:19:39.758 INFO:teuthology.lock.ops:New key found. Updating...
2023-01-14T00:19:39.776 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com
2023-01-14T00:19:39.777 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com
2023-01-14T00:19:39.779 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server
2023-01-14T00:19:39.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com'
2023-01-14T00:19:39.989 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64
2023-01-14T00:19:40.049 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux"
2023-01-14T00:19:40.049 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.6 (Ootpa)"
2023-01-14T00:19:40.050 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel"
2023-01-14T00:19:40.050 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora"
2023-01-14T00:19:40.050 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.6"
2023-01-14T00:19:40.051 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8"
2023-01-14T00:19:40.051 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-01-14T00:19:40.052 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31"
2023-01-14T00:19:40.052 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-01-14T00:19:40.052 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/"
2023-01-14T00:19:40.053 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-01-14T00:19:40.053 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-01-14T00:19:40.053 INFO:teuthology.orchestra.run.smithi119.stdout:
2023-01-14T00:19:40.054 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-01-14T00:19:40.054 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-01-14T00:19:40.054 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-01-14T00:19:40.055 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-01-14T00:19:40.056 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2023-01-14T00:19:40.072 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server
2023-01-14T00:19:40.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-01-14T00:19:40.277 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64
2023-01-14T00:19:40.338 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux"
2023-01-14T00:19:40.338 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)"
2023-01-14T00:19:40.338 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel"
2023-01-14T00:19:40.339 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora"
2023-01-14T00:19:40.339 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6"
2023-01-14T00:19:40.340 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8"
2023-01-14T00:19:40.340 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-01-14T00:19:40.340 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2023-01-14T00:19:40.341 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-01-14T00:19:40.341 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/"
2023-01-14T00:19:40.341 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-01-14T00:19:40.341 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-01-14T00:19:40.342 INFO:teuthology.orchestra.run.smithi167.stdout:
2023-01-14T00:19:40.342 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-01-14T00:19:40.342 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-01-14T00:19:40.343 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-01-14T00:19:40.343 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-01-14T00:19:40.343 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2023-01-14T00:19:40.359 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server
2023-01-14T00:19:40.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com'
2023-01-14T00:19:40.555 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64
2023-01-14T00:19:40.612 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux"
2023-01-14T00:19:40.612 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)"
2023-01-14T00:19:40.612 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel"
2023-01-14T00:19:40.613 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora"
2023-01-14T00:19:40.613 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6"
2023-01-14T00:19:40.613 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8"
2023-01-14T00:19:40.613 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-01-14T00:19:40.613 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31"
2023-01-14T00:19:40.614 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-01-14T00:19:40.614 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/"
2023-01-14T00:19:40.614 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-01-14T00:19:40.614 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-01-14T00:19:40.614 INFO:teuthology.orchestra.run.smithi178.stdout:
2023-01-14T00:19:40.615 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-01-14T00:19:40.615 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-01-14T00:19:40.615 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-01-14T00:19:40.615 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-01-14T00:19:40.616 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2023-01-14T00:19:40.725 INFO:teuthology.dispatcher.supervisor:Running job 7141334
2023-01-14T00:19:40.726 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-01-13_12:08:33-fs-wip-rishabh-testing-2022Nov24-11Jan2023-distro-default-smithi/7141334 --name rishabh-2023-01-13_12:08:33-fs-wip-rishabh-testing-2022Nov24-11Jan2023-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} -- /home/teuthworker/archive/rishabh-2023-01-13_12:08:33-fs-wip-rishabh-testing-2022Nov24-11Jan2023-distro-default-smithi/7141334/orig.config.yaml
2023-01-14T00:19:40.739 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-01-13_12:08:33-fs-wip-rishabh-testing-2022Nov24-11Jan2023-distro-default-smithi/7141334
2023-01-14T00:19:40.740 INFO:teuthology.dispatcher.supervisor:Job PID: 27719
2023-01-14T00:19:40.740 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-01-14T12:20:41.985 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-01-14T12:20:42.177 INFO:teuthology.kill:Killing Pids: {27719}
2023-01-14T12:20:43.011 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0']
2023-01-14T12:20:43.011 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10']
2023-01-14T12:20:43.012 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11']
2023-01-14T12:20:43.012 INFO:teuthology.misc:Compressing logs...
2023-01-14T12:20:43.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com'
2023-01-14T12:20:43.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-01-14T12:20:43.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com'
2023-01-14T12:20:43.966 INFO:teuthology.misc:Compressing logs...
2023-01-14T12:20:44.096 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-01-14T12:20:44.261 INFO:teuthology.misc:Compressing logs...
2023-01-14T12:21:12.616 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2023-01-14T12:21:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.10.log: file size changed while zipping
2023-01-14T12:21:31.192 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mds.h.log: file size changed while zipping
2023-01-14T12:21:32.650 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mds.e.log: file size changed while zipping
2023-01-14T12:21:33.247 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2023-01-14T12:21:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2023-01-14T12:22:07.594 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2023-01-14T12:22:08.615 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2023-01-14T12:22:28.348 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2023-01-14T12:22:28.748 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mgr.x.log: file size changed while zipping
2023-01-14T12:22:35.139 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.11.log: file size changed while zipping
2023-01-14T12:22:37.840 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2023-01-14T12:23:07.913 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.8.log: file size changed while zipping
2023-01-14T12:23:09.347 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mds.f.log: file size changed while zipping
2023-01-14T12:23:11.196 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mds.c.log: file size changed while zipping
2023-01-14T12:23:11.342 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2023-01-14T12:23:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mds.i.log: file size changed while zipping
2023-01-14T12:23:12.872 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.9.log: file size changed while zipping
2023-01-14T12:23:23.244 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2023-01-14T12:23:28.226 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2023-01-14T12:23:30.770 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mds.b.log: file size changed while zipping
2023-01-14T12:23:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2023-01-14T12:23:41.596 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/a043651e-93a2-11ed-821d-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2023-01-14T12:24:19.330 INFO:teuthology.orchestra.run.smithi119.stderr:tar: ./a043651e-93a2-11ed-821d-001a4aab830c/ceph-osd.6.log.gz: file changed as we read it
2023-01-14T12:24:19.330 INFO:teuthology.orchestra.run.smithi119.stderr:tar: ./a043651e-93a2-11ed-821d-001a4aab830c: file changed as we read it
2023-01-14T12:25:15.450 INFO:teuthology.orchestra.run.smithi178.stderr:tar: ./a043651e-93a2-11ed-821d-001a4aab830c/ceph-mds.f.log.gz: file changed as we read it
2023-01-14T12:25:32.208 INFO:teuthology.orchestra.run.smithi178.stderr:tar: ./a043651e-93a2-11ed-821d-001a4aab830c: file changed as we read it
2023-01-14T12:25:32.300 INFO:teuthology.kill:No teuthology processes running
2023-01-14T12:25:44.359 INFO:teuthology.kill:Nuking machines: ['smithi119', 'smithi167', 'smithi178']
2023-01-14T12:25:44.975 INFO:teuthology.kill:2023-01-14 12:25:44,975.975 INFO:teuthology.nuke:targets:
2023-01-14T12:25:44.976 INFO:teuthology.kill:  smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=
2023-01-14T12:25:44.977 INFO:teuthology.kill:  smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=
2023-01-14T12:25:44.977 INFO:teuthology.kill:  smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=
2023-01-14T12:25:44.977 INFO:teuthology.kill:2023-01-14 12:25:44,975.975 INFO:teuthology.nuke:Checking targets against current locks
2023-01-14T12:25:45.036 INFO:teuthology.kill:2023-01-14 12:25:45,035.035 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-14T12:25:45.066 INFO:teuthology.kill:2023-01-14 12:25:45,065.065 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-14T12:25:45.097 INFO:teuthology.kill:2023-01-14 12:25:45,096.096 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-14T12:25:45.129 INFO:teuthology.kill:2023-01-14 12:25:45,129.129 INFO:teuthology.orchestra.console:Power off smithi119
2023-01-14T12:25:45.168 INFO:teuthology.kill:2023-01-14 12:25:45,168.168 INFO:teuthology.orchestra.console:Power off smithi167
2023-01-14T12:25:45.202 INFO:teuthology.kill:2023-01-14 12:25:45,202.202 INFO:teuthology.orchestra.console:Power off smithi178
2023-01-14T12:25:54.504 INFO:teuthology.kill:2023-01-14 12:25:54,503.503 INFO:teuthology.orchestra.console:Power off for smithi119 completed
2023-01-14T12:25:54.560 INFO:teuthology.kill:2023-01-14 12:25:54,560.560 INFO:teuthology.orchestra.console:Power off for smithi178 completed
2023-01-14T12:25:54.665 INFO:teuthology.kill:2023-01-14 12:25:54,665.665 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com
2023-01-14T12:25:54.704 INFO:teuthology.kill:2023-01-14 12:25:54,704.704 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com
2023-01-14T12:26:15.004 INFO:teuthology.kill:2023-01-14 12:26:15,004.004 INFO:teuthology.orchestra.console:Power off for smithi167 completed
2023-01-14T12:26:15.155 INFO:teuthology.kill:2023-01-14 12:26:15,154.154 INFO:teuthology.lock.ops:unlocked: smithi167.front.sepia.ceph.com
2023-01-14T12:28:30.379 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-01-14T12:28:44.337 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi119 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-13_17:05:26-rbd-wip-yuri3-testing-2023-01-12-1538-quincy-distro-default-smithi/7141991
2023-01-14T12:28:44.337 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi178 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-13_17:05:26-rbd-wip-yuri3-testing-2023-01-12-1538-quincy-distro-default-smithi/7141991