2023-05-21T21:55:21.109 INFO:root:teuthology version: 0.0.1.dev100+g01abd1e
2023-05-21T21:55:21.263 INFO:teuthology.lock.ops:Start node 'smithi012.front.sepia.ceph.com' reimaging
2023-05-21T21:55:21.264 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: reset os type and version on server
2023-05-21T21:55:21.264 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server
2023-05-21T21:55:21.294 INFO:teuthology.lock.ops:Node 'smithi012.front.sepia.ceph.com' reimaging is complete
2023-05-21T21:55:21.295 INFO:teuthology.lock.ops:Start node 'smithi067.front.sepia.ceph.com' reimaging
2023-05-21T21:55:21.295 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: reset os type and version on server
2023-05-21T21:55:21.295 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2023-05-21T21:55:21.319 INFO:teuthology.lock.ops:Node 'smithi067.front.sepia.ceph.com' reimaging is complete
2023-05-21T21:55:21.445 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of centos 8.stream
2023-05-21T21:55:21.492 INFO:teuthology.provision.fog.smithi067:Scheduling deploy of centos 8.stream
2023-05-21T21:55:21.655 INFO:teuthology.orchestra.console:Power off smithi012
2023-05-21T21:55:21.700 INFO:teuthology.orchestra.console:Power off smithi067
2023-05-21T21:55:34.154 INFO:teuthology.orchestra.console:Power off for smithi012 completed
2023-05-21T21:55:34.186 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2023-05-21T21:55:34.255 INFO:teuthology.orchestra.console:Power on smithi012
2023-05-21T21:55:34.287 INFO:teuthology.orchestra.console:Power on smithi067
2023-05-21T21:55:38.538 INFO:teuthology.orchestra.console:Power on for smithi067 completed
2023-05-21T21:55:38.639 INFO:teuthology.provision.fog.smithi067:Waiting for deploy to finish
2023-05-21T21:55:42.623 INFO:teuthology.orchestra.console:Power on for smithi012 completed
2023-05-21T21:55:42.724 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish
2023-05-21T21:58:31.792 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12
2023-05-21T21:58:40.912 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12
2023-05-21T21:58:42.800 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.67
2023-05-21T21:58:49.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12
2023-05-21T21:58:51.856 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.67
2023-05-21T21:58:59.024 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12
2023-05-21T21:59:00.912 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.67
2023-05-21T21:59:08.080 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.12
2023-05-21T21:59:09.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.67
2023-05-21T21:59:19.024 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.67
2023-05-21T21:59:28.080 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.67
2023-05-21T22:00:14.123 WARNING:teuthology.provision.fog:timed out
2023-05-21T22:00:20.466 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-21T22:00:20.735 INFO:teuthology.provision.fog.smithi012:Node is ready
2023-05-21T22:00:20.754 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com
2023-05-21T22:00:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012
2023-05-21T22:00:21.169 INFO:teuthology.provision.fog.smithi012:Deploy complete!
2023-05-21T22:00:34.091 WARNING:teuthology.provision.fog:timed out
2023-05-21T22:00:40.321 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-21T22:00:40.583 INFO:teuthology.provision.fog.smithi067:Node is ready
2023-05-21T22:00:40.604 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067.front.sepia.ceph.com
2023-05-21T22:00:40.665 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67 smithi067.front.sepia.ceph.com smithi067
2023-05-21T22:00:41.002 INFO:teuthology.provision.fog.smithi067:Deploy complete!
2023-05-21T22:00:41.227 INFO:teuthology.lock.ops:Checking smithi012.front.sepia.ceph.com
2023-05-21T22:00:41.228 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-21T22:00:41.252 INFO:teuthology.lock.ops:Checking smithi067.front.sepia.ceph.com
2023-05-21T22:00:41.253 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-21T22:00:41.279 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: set os type and version on server
2023-05-21T22:00:41.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com'
2023-05-21T22:00:41.547 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64
2023-05-21T22:00:41.607 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream"
2023-05-21T22:00:41.608 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8"
2023-05-21T22:00:41.608 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos"
2023-05-21T22:00:41.608 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora"
2023-05-21T22:00:41.608 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8"
2023-05-21T22:00:41.609 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8"
2023-05-21T22:00:41.609 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 8"
2023-05-21T22:00:41.609 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31"
2023-05-21T22:00:41.609 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-05-21T22:00:41.610 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/"
2023-05-21T22:00:41.610 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-21T22:00:41.610 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-21T22:00:41.610 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-05-21T22:00:41.611 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server
2023-05-21T22:00:41.636 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: set os type and version on server
2023-05-21T22:00:41.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com'
2023-05-21T22:00:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64
2023-05-21T22:00:41.888 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream"
2023-05-21T22:00:41.889 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8"
2023-05-21T22:00:41.889 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos"
2023-05-21T22:00:41.889 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora"
2023-05-21T22:00:41.889 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8"
2023-05-21T22:00:41.889 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8"
2023-05-21T22:00:41.889 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 8"
2023-05-21T22:00:41.890 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31"
2023-05-21T22:00:41.890 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-05-21T22:00:41.890 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/"
2023-05-21T22:00:41.890 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-21T22:00:41.890 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-21T22:00:41.890 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-05-21T22:00:41.891 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2023-05-21T22:00:42.014 INFO:teuthology.dispatcher.supervisor:Running job 7281968
2023-05-21T22:00:42.014 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01abd1ed90624b09fd81d9cb3a79e2834351f0ca/virtualenv/bin/teuthology -v --owner scheduled_sjust@teuthology --archive /home/teuthworker/archive/sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi/7281968 --name sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi --description crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/pool-snaps-few-objects} -- /home/teuthworker/archive/sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi/7281968/orig.config.yaml
2023-05-21T22:00:42.017 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi/7281968
2023-05-21T22:00:42.018 INFO:teuthology.dispatcher.supervisor:Job PID: 27352
2023-05-21T22:00:42.019 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-05-22T10:02:36.522 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-05-22T10:02:41.507 INFO:teuthology.kill:Killing Pids: {27352}
2023-05-22T10:02:47.186 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a']
2023-05-22T10:02:47.189 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']
2023-05-22T10:02:47.197 INFO:teuthology.misc:Compressing logs...
2023-05-22T10:02:47.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com'
2023-05-22T10:02:54.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com'
2023-05-22T10:02:54.788 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-05-22T10:02:54.795 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-05-22T10:02:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	/home/ubuntu/cephtest/archive/syslog/misc.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-05-22T10:02:54.796 INFO:teuthology.orchestra.run.smithi067.stderr: 83.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-05-22T10:02:55.514 INFO:teuthology.kill:No teuthology processes running
2023-05-22T10:02:57.814 INFO:teuthology.kill:Nuking machines: ['smithi012', 'smithi067']
2023-05-22T10:02:58.460 INFO:teuthology.kill:2023-05-22 10:02:58,460.460 INFO:teuthology.nuke:targets:
2023-05-22T10:02:58.461 INFO:teuthology.kill:  smithi012.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+bh1MLeMtOQ6hduoiZhr0csQ+vUMOA4ny3k0NkIm3WSImRI7WAdq2gkwe0QCXosYy1qWny3KR0ctM2RQNIn3UmX3pSVflgmoOL7SF8z4ZfyLgWCDcn21sQ9jIbtHI+QVjuwX1QyKGlWQH3N3F1X2HIDuabOgfz+upKLl9HWKJPE7u5VW6jB2OCsLQ+otqS1IwEqHh12+rM5cLRQeD9hXsmGyeapyhno5kOrJYCzfMRTyZcDJCOmXBi8dH94bcGcr+cTigt70UZgn/nxCZWpw2WsNz9gMVvbVhtDUwoiR+0otVr4L/SziteX5A0qcebQiJ146d/xoymLiKbI1Vl8MbEsefzKdeAFZPkuP8142X6KrqtBqQrpMhZkKs5Ila4+NpUz8ugPbYOt8gi6vqlEXhuTBBGm7pV2iWd/1bKje/tgRfO2gn7gjX0tKUTssphvWPylzQU1C2CGC1/cXUNR2XTcjNDw2eZ5wFYDqQ8m5yYJQ1hNUaWc/79b5Xc0QfDmk=
2023-05-22T10:02:58.462 INFO:teuthology.kill:  smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgq+0p92cd4lVeyc1b10pBFJTIJjiQAsXeMZ+gLifMzCcYaI6GTei5utIRzaC3kFAKlkLuYPc3Myov3VwkBtjE=
2023-05-22T10:02:58.462 INFO:teuthology.kill:2023-05-22 10:02:58,460.460 INFO:teuthology.nuke:Checking targets against current locks
2023-05-22T10:02:58.521 INFO:teuthology.kill:2023-05-22 10:02:58,521.521 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-05-22T10:02:58.555 INFO:teuthology.kill:2023-05-22 10:02:58,555.555 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-05-22T10:02:58.609 INFO:teuthology.kill:2023-05-22 10:02:58,609.609 INFO:teuthology.orchestra.console:Power off smithi012
2023-05-22T10:02:58.650 INFO:teuthology.kill:2023-05-22 10:02:58,649.649 INFO:teuthology.orchestra.console:Power off smithi067
2023-05-22T10:03:06.985 INFO:teuthology.kill:2023-05-22 10:03:06,984.984 INFO:teuthology.orchestra.console:Power off for smithi012 completed
2023-05-22T10:03:07.011 INFO:teuthology.kill:2023-05-22 10:03:07,011.011 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2023-05-22T10:03:07.160 INFO:teuthology.kill:2023-05-22 10:03:07,160.160 INFO:teuthology.lock.ops:unlocked: smithi012.front.sepia.ceph.com
2023-05-22T10:03:07.168 INFO:teuthology.kill:2023-05-22 10:03:07,168.168 INFO:teuthology.lock.ops:unlocked: smithi067.front.sepia.ceph.com
2023-05-22T10:05:07.605 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-05-22T10:05:07.815 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi012 but it was locked by another job: /home/teuthworker/archive/adking-2023-05-22_01:46:07-orch:cephadm-wip-adk2-testing-2023-05-21-1627-reef-distro-default-smithi/7282291
2023-05-22T10:05:07.815 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi067 but it was locked by another job: /home/teuthworker/archive/adking-2023-05-22_01:46:07-orch:cephadm-wip-adk2-testing-2023-05-21-1627-reef-distro-default-smithi/7282291