2023-03-16T20:08:00.193 INFO:root:teuthology version: 0.0.1.dev57+g5c9057f
2023-03-16T20:08:01.687 INFO:teuthology.lock.ops:Start node 'smithi093.front.sepia.ceph.com' reimaging
2023-03-16T20:08:03.838 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: reset os type and version on server
2023-03-16T20:08:03.838 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2023-03-16T20:08:03.862 INFO:teuthology.lock.ops:Node 'smithi093.front.sepia.ceph.com' reimaging is complete
2023-03-16T20:08:07.542 INFO:teuthology.lock.ops:Start node 'smithi101.front.sepia.ceph.com' reimaging
2023-03-16T20:08:07.542 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: reset os type and version on server
2023-03-16T20:08:07.542 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2023-03-16T20:08:08.379 INFO:teuthology.lock.ops:Node 'smithi101.front.sepia.ceph.com' reimaging is complete
2023-03-16T20:08:10.179 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging
2023-03-16T20:08:10.179 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server
2023-03-16T20:08:10.179 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2023-03-16T20:08:10.204 INFO:teuthology.lock.ops:Node 'smithi178.front.sepia.ceph.com' reimaging is complete
2023-03-16T20:08:11.393 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of rhel 8.6
2023-03-16T20:08:12.333 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of rhel 8.6
2023-03-16T20:08:13.407 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of rhel 8.6
2023-03-16T20:08:14.935 INFO:teuthology.orchestra.console:Power off smithi101
2023-03-16T20:08:15.997 INFO:teuthology.orchestra.console:Power off smithi093
2023-03-16T20:08:17.081 INFO:teuthology.orchestra.console:Power off smithi178
2023-03-16T20:08:18.471 INFO:teuthology.orchestra.console:Power off for smithi101 completed
2023-03-16T20:08:19.848 INFO:teuthology.orchestra.console:Power on smithi101
2023-03-16T20:08:21.658 INFO:teuthology.orchestra.console:Power off for smithi178 completed
2023-03-16T20:08:23.277 INFO:teuthology.orchestra.console:Power on smithi178
2023-03-16T20:08:27.418 INFO:teuthology.orchestra.console:Power on for smithi101 completed
2023-03-16T20:08:28.550 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish
2023-03-16T20:08:29.946 INFO:teuthology.orchestra.console:Power off for smithi093 completed
2023-03-16T20:08:31.709 INFO:teuthology.orchestra.console:Power on for smithi178 completed
2023-03-16T20:08:31.810 INFO:teuthology.orchestra.console:Power on smithi093
2023-03-16T20:08:33.188 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish
2023-03-16T20:08:38.826 INFO:teuthology.orchestra.console:Power on for smithi093 completed
2023-03-16T20:08:39.587 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish
2023-03-16T20:11:18.085 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.93
2023-03-16T20:11:20.135 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-16T20:11:29.189 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.93
2023-03-16T20:11:29.852 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-16T20:11:38.405 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-03-16T20:11:40.407 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-16T20:11:40.408 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.93
2023-03-16T20:11:49.477 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-16T20:11:51.776 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.93
2023-03-16T20:11:51.776 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-03-16T20:12:00.837 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.93
2023-03-16T20:12:02.034 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.101
2023-03-16T20:12:02.053 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-03-16T20:12:11.109 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.93
2023-03-16T20:12:12.549 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-03-16T20:12:21.605 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-03-16T20:12:33.349 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178
2023-03-16T20:12:39.934 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-16T20:12:41.460 INFO:teuthology.provision.fog.smithi101:Node is ready
2023-03-16T20:12:42.905 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com
2023-03-16T20:12:44.126 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101
2023-03-16T20:12:47.918 INFO:teuthology.provision.fog.smithi101:Deploy complete!
2023-03-16T20:12:50.390 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-16T20:12:51.142 INFO:teuthology.provision.fog.smithi093:Node is ready
2023-03-16T20:12:51.686 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com
2023-03-16T20:12:52.619 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093
2023-03-16T20:12:54.371 INFO:teuthology.provision.fog.smithi093:Deploy complete!
2023-03-16T20:13:41.413 WARNING:teuthology.provision.fog:timed out
2023-03-16T20:13:48.561 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-16T20:13:50.522 INFO:teuthology.provision.fog.smithi178:Node is ready
2023-03-16T20:13:52.358 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com
2023-03-16T20:13:53.922 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178
2023-03-16T20:13:55.333 INFO:teuthology.provision.fog.smithi178:Deploy complete!
2023-03-16T20:13:56.766 INFO:teuthology.lock.ops:Checking smithi093.front.sepia.ceph.com
2023-03-16T20:13:58.777 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-16T20:13:58.806 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com
2023-03-16T20:14:00.226 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-16T20:14:00.247 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com
2023-03-16T20:14:02.882 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: set os type and version on server
2023-03-16T20:14:02.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com'
2023-03-16T20:14:03.507 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64
2023-03-16T20:14:03.852 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/"
2023-03-16T20:14:04.945 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-16T20:14:04.946 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-16T20:14:04.946 INFO:teuthology.orchestra.run.smithi093.stdout:
2023-03-16T20:14:04.946 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-16T20:14:04.946 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-16T20:14:04.946 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-16T20:14:04.946 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-16T20:14:04.947 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2023-03-16T20:14:04.980 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server
2023-03-16T20:14:05.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com'
2023-03-16T20:14:06.172 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64
2023-03-16T20:14:06.890 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Red Hat Enterprise Linux"
2023-03-16T20:14:08.815 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8.6 (Ootpa)"
2023-03-16T20:14:08.816 INFO:teuthology.orchestra.run.smithi101.stdout:ID="rhel"
2023-03-16T20:14:08.816 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="fedora"
2023-03-16T20:14:08.816 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8.6"
2023-03-16T20:14:08.816 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8"
2023-03-16T20:14:08.816 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-16T20:14:10.582 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31"
2023-03-16T20:14:10.582 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-16T20:14:10.582 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.redhat.com/"
2023-03-16T20:14:10.582 INFO:teuthology.orchestra.run.smithi101.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-16T20:14:10.582 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-16T20:14:10.583 INFO:teuthology.orchestra.run.smithi101.stdout:
2023-03-16T20:14:10.583 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-16T20:14:10.583 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-16T20:14:10.583 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-16T20:14:10.583 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-16T20:14:10.583 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server
2023-03-16T20:14:10.604 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server
2023-03-16T20:14:13.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com'
2023-03-16T20:14:13.571 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64
2023-03-16T20:14:16.201 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux"
2023-03-16T20:14:18.598 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)"
2023-03-16T20:14:18.598 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel"
2023-03-16T20:14:18.598 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora"
2023-03-16T20:14:18.598 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6"
2023-03-16T20:14:18.598 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8"
2023-03-16T20:14:18.598 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-16T20:14:18.598 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31"
2023-03-16T20:14:18.599 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-16T20:14:18.599 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/"
2023-03-16T20:14:18.599 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-16T20:14:18.599 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-16T20:14:18.599 INFO:teuthology.orchestra.run.smithi178.stdout:
2023-03-16T20:14:21.245 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-16T20:14:21.245 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-16T20:14:21.245 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-16T20:14:21.245 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-16T20:14:21.246 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2023-03-16T20:14:21.363 INFO:teuthology.dispatcher.supervisor:Running job 7210405
2023-03-16T20:14:23.019 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2023-03-16_19:02:23-fs:workload-wip-pdonnell-testing-20230316.161018-distro-default-smithi/7210405 --name pdonnell-2023-03-16_19:02:23-fs:workload-wip-pdonnell-testing-20230316.161018-distro-default-smithi --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}} -- /home/teuthworker/archive/pdonnell-2023-03-16_19:02:23-fs:workload-wip-pdonnell-testing-20230316.161018-distro-default-smithi/7210405/orig.config.yaml
2023-03-16T20:14:23.023 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2023-03-16_19:02:23-fs:workload-wip-pdonnell-testing-20230316.161018-distro-default-smithi/7210405
2023-03-16T20:14:24.875 INFO:teuthology.dispatcher.supervisor:Job PID: 29538
2023-03-16T20:14:24.875 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-16T21:50:31.403 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1