2023-05-21T23:21:35.079 INFO:root:teuthology version: 0.0.1.dev101+gc6e0984
2023-05-21T23:21:35.770 INFO:teuthology.lock.ops:Start node 'smithi066.front.sepia.ceph.com' reimaging
2023-05-21T23:21:35.771 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: reset os type and version on server
2023-05-21T23:21:35.771 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server
2023-05-21T23:21:36.814 INFO:teuthology.lock.ops:Node 'smithi066.front.sepia.ceph.com' reimaging is complete
2023-05-21T23:21:36.814 INFO:teuthology.lock.ops:Start node 'smithi123.front.sepia.ceph.com' reimaging
2023-05-21T23:21:36.814 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: reset os type and version on server
2023-05-21T23:21:36.814 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2023-05-21T23:21:36.840 INFO:teuthology.lock.ops:Node 'smithi123.front.sepia.ceph.com' reimaging is complete
2023-05-21T23:21:36.979 INFO:teuthology.provision.fog.smithi066:Scheduling deploy of centos 8.stream
2023-05-21T23:21:37.001 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of centos 8.stream
2023-05-21T23:21:37.214 INFO:teuthology.orchestra.console:Power off smithi066
2023-05-21T23:21:37.237 INFO:teuthology.orchestra.console:Power off smithi123
2023-05-21T23:21:49.769 INFO:teuthology.orchestra.console:Power off for smithi066 completed
2023-05-21T23:21:49.777 INFO:teuthology.orchestra.console:Power off for smithi123 completed
2023-05-21T23:21:49.878 INFO:teuthology.orchestra.console:Power on smithi066
2023-05-21T23:21:49.886 INFO:teuthology.orchestra.console:Power on smithi123
2023-05-21T23:21:54.140 INFO:teuthology.orchestra.console:Power on for smithi123 completed
2023-05-21T23:21:54.241 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish
2023-05-21T23:21:58.247 INFO:teuthology.orchestra.console:Power on for smithi066 completed
2023-05-21T23:21:58.348 INFO:teuthology.provision.fog.smithi066:Waiting for deploy to finish
2023-05-21T23:24:58.609 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-05-21T23:25:07.728 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-05-21T23:25:16.816 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-05-21T23:25:17.552 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-05-21T23:25:25.904 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-05-21T23:25:26.608 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-05-21T23:25:35.664 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-05-21T23:25:44.784 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-05-21T23:25:53.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-05-21T23:26:03.088 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-05-21T23:26:12.208 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-05-21T23:26:31.927 WARNING:teuthology.provision.fog:timed out
2023-05-21T23:26:38.175 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-21T23:26:38.434 INFO:teuthology.provision.fog.smithi123:Node is ready
2023-05-21T23:26:38.495 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com
2023-05-21T23:26:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123
2023-05-21T23:26:38.925 INFO:teuthology.provision.fog.smithi123:Deploy complete!
2023-05-21T23:27:18.444 WARNING:teuthology.provision.fog:timed out
2023-05-21T23:27:25.099 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-21T23:27:25.335 INFO:teuthology.provision.fog.smithi066:Node is ready
2023-05-21T23:27:25.355 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066.front.sepia.ceph.com
2023-05-21T23:27:25.417 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66 smithi066.front.sepia.ceph.com smithi066
2023-05-21T23:27:25.758 INFO:teuthology.provision.fog.smithi066:Deploy complete!
2023-05-21T23:27:26.042 INFO:teuthology.lock.ops:Checking smithi066.front.sepia.ceph.com
2023-05-21T23:27:26.072 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-21T23:27:26.102 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com
2023-05-21T23:27:26.102 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-21T23:27:26.128 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: set os type and version on server
2023-05-21T23:27:26.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com'
2023-05-21T23:27:26.323 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64
2023-05-21T23:27:26.382 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream"
2023-05-21T23:27:26.382 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8"
2023-05-21T23:27:26.382 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos"
2023-05-21T23:27:26.382 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora"
2023-05-21T23:27:26.382 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8"
2023-05-21T23:27:26.383 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8"
2023-05-21T23:27:26.383 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 8"
2023-05-21T23:27:26.383 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31"
2023-05-21T23:27:26.383 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-05-21T23:27:26.383 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/"
2023-05-21T23:27:26.383 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-21T23:27:26.383 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-21T23:27:26.384 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-05-21T23:27:26.385 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server
2023-05-21T23:27:26.410 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server
2023-05-21T23:27:26.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2023-05-21T23:27:26.668 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64
2023-05-21T23:27:26.727 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream"
2023-05-21T23:27:26.727 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8"
2023-05-21T23:27:26.728 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos"
2023-05-21T23:27:26.728 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora"
2023-05-21T23:27:26.728 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8"
2023-05-21T23:27:26.728 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8"
2023-05-21T23:27:26.728 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 8"
2023-05-21T23:27:26.728 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31"
2023-05-21T23:27:26.729 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-05-21T23:27:26.729 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/"
2023-05-21T23:27:26.729 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-21T23:27:26.729 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-21T23:27:26.729 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-05-21T23:27:26.730 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2023-05-21T23:27:26.866 INFO:teuthology.dispatcher.supervisor:Running job 7281958
2023-05-21T23:27:26.867 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2023-05-21_19:06:26-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7281958 --name pdonnell-2023-05-21_19:06:26-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/pdonnell-2023-05-21_19:06:26-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7281958/orig.config.yaml
2023-05-21T23:27:26.885 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2023-05-21_19:06:26-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7281958
2023-05-21T23:27:26.886 INFO:teuthology.dispatcher.supervisor:Job PID: 24390
2023-05-21T23:27:26.886 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-05-22T11:27:35.188 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-05-22T11:27:41.973 INFO:teuthology.kill:Killing Pids: {24390}
2023-05-22T11:27:44.945 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2023-05-22T11:27:44.954 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2023-05-22T11:27:45.035 INFO:teuthology.misc:Compressing logs...
2023-05-22T11:27:45.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com'
2023-05-22T11:27:51.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2023-05-22T11:27:51.082 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-05-22T11:27:51.113 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-05-22T11:27:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-05-22T11:27:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	 83.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-05-22T11:27:51.559 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-05-22T11:27:51.560 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-05-22T11:27:51.560 INFO:teuthology.orchestra.run.smithi123.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	 80.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-05-22T11:27:51.568 INFO:teuthology.orchestra.run.smithi123.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	 87.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-05-22T11:27:52.922 INFO:teuthology.kill:No teuthology processes running
2023-05-22T11:27:53.712 INFO:teuthology.kill:No locked machines. Not nuking anything
2023-05-22T11:29:54.025 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-05-22T11:29:54.907 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi066 but it was locked by another job: /home/teuthworker/archive/pdonnell-2023-05-22_01:10:22-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7282201
2023-05-22T11:29:54.907 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi123 but it was locked by another job: /home/teuthworker/archive/soumyakoduri-2023-05-22_10:03:38-rgw:multisite-wip-skoduri-bucket-repl-distro-default-smithi/7282402