2023-03-16T08:29:56.177 INFO:root:teuthology version: 0.0.1.dev57+g5c9057f
2023-03-16T08:29:57.808 INFO:teuthology.lock.ops:Start node 'smithi060.front.sepia.ceph.com' reimaging
2023-03-16T08:29:58.559 INFO:teuthology.lock.ops:Updating [smithi060.front.sepia.ceph.com]: reset os type and version on server
2023-03-16T08:30:00.397 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server
2023-03-16T08:30:00.422 INFO:teuthology.lock.ops:Node 'smithi060.front.sepia.ceph.com' reimaging is complete
2023-03-16T08:30:01.756 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging
2023-03-16T08:30:01.757 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server
2023-03-16T08:30:01.757 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server
2023-03-16T08:30:01.780 INFO:teuthology.lock.ops:Node 'smithi072.front.sepia.ceph.com' reimaging is complete
2023-03-16T08:30:02.954 INFO:teuthology.lock.ops:Start node 'smithi116.front.sepia.ceph.com' reimaging
2023-03-16T08:30:02.954 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: reset os type and version on server
2023-03-16T08:30:02.954 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2023-03-16T08:30:02.980 INFO:teuthology.lock.ops:Node 'smithi116.front.sepia.ceph.com' reimaging is complete
2023-03-16T08:30:04.457 INFO:teuthology.provision.fog.smithi060:Scheduling deploy of rhel 8.6
2023-03-16T08:30:05.783 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of rhel 8.6
2023-03-16T08:30:06.776 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of rhel 8.6
2023-03-16T08:30:07.831 INFO:teuthology.orchestra.console:Power off smithi072
2023-03-16T08:30:08.982 INFO:teuthology.orchestra.console:Power off smithi060
2023-03-16T08:30:10.197 INFO:teuthology.orchestra.console:Power off smithi116
2023-03-16T08:30:20.870 INFO:teuthology.orchestra.console:Power off for smithi072 completed
2023-03-16T08:30:21.752 INFO:teuthology.orchestra.console:Power off for smithi060 completed
2023-03-16T08:30:21.852 INFO:teuthology.orchestra.console:Power on smithi072
2023-03-16T08:30:23.044 INFO:teuthology.orchestra.console:Power on smithi060
2023-03-16T08:30:23.887 INFO:teuthology.orchestra.console:Power off for smithi116 completed
2023-03-16T08:30:24.906 INFO:teuthology.orchestra.console:Power on smithi116
2023-03-16T08:30:29.042 INFO:teuthology.orchestra.console:Power on for smithi072 completed
2023-03-16T08:30:29.757 INFO:teuthology.orchestra.console:Power on for smithi060 completed
2023-03-16T08:30:29.858 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish
2023-03-16T08:30:30.417 INFO:teuthology.provision.fog.smithi060:Waiting for deploy to finish
2023-03-16T08:30:32.840 INFO:teuthology.orchestra.console:Power on for smithi116 completed
2023-03-16T08:30:34.462 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish
2023-03-16T08:33:06.373 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.60
2023-03-16T08:33:16.389 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.60
2023-03-16T08:33:24.869 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.116
2023-03-16T08:33:30.821 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.60
2023-03-16T08:33:39.365 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.116
2023-03-16T08:33:40.439 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-03-16T08:33:45.381 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.60
2023-03-16T08:33:49.513 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.116
2023-03-16T08:33:50.715 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-03-16T08:33:59.781 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.116
2023-03-16T08:34:00.905 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-03-16T08:34:09.989 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-03-16T08:34:19.397 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-03-16T08:34:28.965 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-03-16T08:34:51.995 WARNING:teuthology.provision.fog:timed out
2023-03-16T08:35:00.484 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-16T08:35:01.494 INFO:teuthology.provision.fog.smithi060:Node is ready
2023-03-16T08:35:02.557 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060.front.sepia.ceph.com
2023-03-16T08:35:05.077 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60 smithi060.front.sepia.ceph.com smithi060
2023-03-16T08:35:07.001 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-03-16T08:35:08.974 WARNING:teuthology.provision.fog:timed out
2023-03-16T08:35:10.704 INFO:teuthology.provision.fog.smithi060:Deploy complete!
2023-03-16T08:35:33.477 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-16T08:35:34.804 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-16T08:35:36.236 INFO:teuthology.provision.fog.smithi116:Node is ready
2023-03-16T08:35:37.487 INFO:teuthology.provision.fog.smithi072:Node is ready
2023-03-16T08:35:37.504 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116
2023-03-16T08:35:39.162 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com
2023-03-16T08:35:39.180 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116
2023-03-16T08:35:41.019 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072
2023-03-16T08:35:42.871 INFO:teuthology.provision.fog.smithi116:Deploy complete!
2023-03-16T08:35:44.090 INFO:teuthology.provision.fog.smithi072:Deploy complete!
2023-03-16T08:35:45.714 INFO:teuthology.lock.ops:Checking smithi060.front.sepia.ceph.com
2023-03-16T08:35:46.754 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-16T08:35:48.190 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com
2023-03-16T08:35:49.551 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-16T08:35:49.572 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com
2023-03-16T08:35:50.249 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-16T08:35:50.274 INFO:teuthology.lock.ops:Updating [smithi060.front.sepia.ceph.com]: set os type and version on server
2023-03-16T08:35:52.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com'
2023-03-16T08:35:52.525 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64
2023-03-16T08:36:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="Red Hat Enterprise Linux"
2023-03-16T08:36:04.269 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8.6 (Ootpa)"
2023-03-16T08:36:04.269 INFO:teuthology.orchestra.run.smithi060.stdout:ID="rhel"
2023-03-16T08:36:04.269 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="fedora"
2023-03-16T08:36:04.270 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8.6"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://www.redhat.com/"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-16T08:36:04.271 INFO:teuthology.orchestra.run.smithi060.stdout:
2023-03-16T08:36:04.767 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-16T08:36:04.767 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-16T08:36:05.421 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-16T08:36:05.422 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-16T08:36:05.424 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server
2023-03-16T08:36:05.454 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server
2023-03-16T08:36:05.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com'
2023-03-16T08:36:06.171 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64
2023-03-16T08:36:16.965 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Red Hat Enterprise Linux"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8.6 (Ootpa)"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:ID="rhel"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="fedora"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8.6"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.redhat.com/"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-16T08:36:18.363 INFO:teuthology.orchestra.run.smithi072.stdout:
2023-03-16T08:36:19.710 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-16T08:36:19.710 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-16T08:36:19.711 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-16T08:36:19.711 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-16T08:36:19.712 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server
2023-03-16T08:36:19.735 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server
2023-03-16T08:36:20.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com'
2023-03-16T08:36:20.951 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64
2023-03-16T08:36:22.450 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Red Hat Enterprise Linux"
2023-03-16T08:36:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8.6 (Ootpa)"
2023-03-16T08:36:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:ID="rhel"
2023-03-16T08:36:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="fedora"
2023-03-16T08:36:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8.6"
2023-03-16T08:36:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.redhat.com/"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-16T08:36:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-03-16T08:36:24.885 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-03-16T08:36:24.885 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-03-16T08:36:24.886 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server
2023-03-16T08:36:25.018 INFO:teuthology.dispatcher.supervisor:Running job 7208960
2023-03-16T08:36:26.121 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5c9057f4b1ecae7b7410e418e40b739ff0616d25/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2023-03-15_20:41:39-fs:workload-wip-pdonnell-testing-20230315.190337-distro-default-smithi/7208960 --name pdonnell-2023-03-15_20:41:39-fs:workload-wip-pdonnell-testing-20230315.190337-distro-default-smithi --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} -- /home/teuthworker/archive/pdonnell-2023-03-15_20:41:39-fs:workload-wip-pdonnell-testing-20230315.190337-distro-default-smithi/7208960/orig.config.yaml
2023-03-16T08:36:26.143 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2023-03-15_20:41:39-fs:workload-wip-pdonnell-testing-20230315.190337-distro-default-smithi/7208960
2023-03-16T08:36:27.361 INFO:teuthology.dispatcher.supervisor:Job PID: 17065
2023-03-16T08:36:27.361 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-16T14:02:50.680 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-03-16T14:02:53.747 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi060 but it was locked by another job: /home/teuthworker/archive/xiubli-2023-03-16_11:15:12-fs:workload-wip-20230316-1328-unlink-testing-default-smithi/7210194
2023-03-16T14:02:55.696 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi072 but it was locked by another job: /home/teuthworker/archive/nmordech-2023-03-16_11:19:56-rados:perf-main-distro-default-smithi/7210198
2023-03-16T14:02:55.697 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi116 but it was locked by another job: /home/teuthworker/archive/akupczyk-2023-03-15_22:15:25-rados-wip-yuri8-testing-2023-03-07-1527-distro-default-smithi/7209161