2023-05-21T22:24:42.234 INFO:root:teuthology version: 0.0.1.dev100+g01abd1e
2023-05-21T22:24:42.748 INFO:teuthology.lock.ops:Start node 'smithi038.front.sepia.ceph.com' reimaging
2023-05-21T22:24:42.749 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: reset os type and version on server
2023-05-21T22:24:42.749 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2023-05-21T22:24:42.777 INFO:teuthology.lock.ops:Node 'smithi038.front.sepia.ceph.com' reimaging is complete
2023-05-21T22:24:42.778 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging
2023-05-21T22:24:42.778 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server
2023-05-21T22:24:42.778 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2023-05-21T22:24:42.802 INFO:teuthology.lock.ops:Node 'smithi164.front.sepia.ceph.com' reimaging is complete
2023-05-21T22:24:42.930 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 8.stream
2023-05-21T22:24:42.948 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 8.stream
2023-05-21T22:24:43.143 INFO:teuthology.orchestra.console:Power off smithi164
2023-05-21T22:24:43.208 INFO:teuthology.orchestra.console:Power off smithi038
2023-05-21T22:24:51.853 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2023-05-21T22:24:51.860 INFO:teuthology.orchestra.console:Power off for smithi164 completed
2023-05-21T22:24:51.955 INFO:teuthology.orchestra.console:Power on smithi038
2023-05-21T22:24:51.965 INFO:teuthology.orchestra.console:Power on smithi164
2023-05-21T22:24:56.216 INFO:teuthology.orchestra.console:Power on for smithi164 completed
2023-05-21T22:24:56.317 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2023-05-21T22:25:00.337 INFO:teuthology.orchestra.console:Power on for smithi038 completed
2023-05-21T22:25:00.438 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2023-05-21T22:27:49.512 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-05-21T22:27:58.640 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-05-21T22:28:00.496 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-05-21T22:28:07.696 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-05-21T22:28:09.552 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-05-21T22:28:16.752 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-05-21T22:28:18.608 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-05-21T22:28:25.840 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-05-21T22:28:27.696 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-05-21T22:28:34.896 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-05-21T22:28:36.752 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-05-21T22:28:43.952 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-05-21T22:29:42.811 WARNING:teuthology.provision.fog:timed out
2023-05-21T22:29:49.317 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-21T22:29:49.537 INFO:teuthology.provision.fog.smithi164:Node is ready
2023-05-21T22:29:49.557 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com
2023-05-21T22:29:49.617 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164
2023-05-21T22:29:49.955 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2023-05-21T22:29:49.961 WARNING:teuthology.provision.fog:timed out
2023-05-21T22:29:56.212 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-05-21T22:29:56.438 INFO:teuthology.provision.fog.smithi038:Node is ready
2023-05-21T22:29:56.458 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2023-05-21T22:29:56.518 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2023-05-21T22:29:56.859 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2023-05-21T22:29:57.232 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2023-05-21T22:29:57.297 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-21T22:29:57.318 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com
2023-05-21T22:29:57.319 INFO:teuthology.lock.ops:New key found. Updating...
2023-05-21T22:29:57.345 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2023-05-21T22:29:57.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2023-05-21T22:29:57.546 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2023-05-21T22:29:57.606 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2023-05-21T22:29:57.606 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8"
2023-05-21T22:29:57.606 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2023-05-21T22:29:57.606 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 8"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-21T22:29:57.607 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-21T22:29:57.608 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-05-21T22:29:57.609 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2023-05-21T22:29:57.636 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server
2023-05-21T22:29:57.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2023-05-21T22:29:57.838 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64
2023-05-21T22:29:57.898 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2023-05-21T22:29:57.899 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8"
2023-05-21T22:29:57.899 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2023-05-21T22:29:57.899 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2023-05-21T22:29:57.899 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8"
2023-05-21T22:29:57.899 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8"
2023-05-21T22:29:57.899 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8"
2023-05-21T22:29:57.899 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2023-05-21T22:29:57.900 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-05-21T22:29:57.900 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2023-05-21T22:29:57.900 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-05-21T22:29:57.900 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-05-21T22:29:57.900 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-05-21T22:29:57.901 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2023-05-21T22:29:58.047 INFO:teuthology.dispatcher.supervisor:Running job 7281977
2023-05-21T22:29:58.048 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01abd1ed90624b09fd81d9cb3a79e2834351f0ca/virtualenv/bin/teuthology -v --owner scheduled_sjust@teuthology --archive /home/teuthworker/archive/sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi/7281977 --name sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi --description crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} -- /home/teuthworker/archive/sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi/7281977/orig.config.yaml
2023-05-21T22:29:58.104 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sjust-2023-05-21_21:38:43-crimson-rados-wip-matanb-crimson-testing-21.5-reef-distro-default-smithi/7281977
2023-05-21T22:29:58.105 INFO:teuthology.dispatcher.supervisor:Job PID: 22630
2023-05-21T22:29:58.106 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-05-22T10:30:05.491 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-05-22T10:30:10.225 INFO:teuthology.kill:Killing Pids: {22630}
2023-05-22T10:30:15.719 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a']
2023-05-22T10:30:15.734 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']
2023-05-22T10:30:15.770 INFO:teuthology.misc:Compressing logs...
2023-05-22T10:30:15.907 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2023-05-22T10:30:21.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2023-05-22T10:30:21.916 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2023-05-22T10:30:21.934 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2023-05-22T10:30:21.934 INFO:teuthology.orchestra.run.smithi164.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2023-05-22T10:30:21.934 INFO:teuthology.orchestra.run.smithi164.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	 83.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2023-05-22T10:30:23.194 INFO:teuthology.kill:No teuthology processes running
2023-05-22T10:30:24.669 INFO:teuthology.kill:Nuking machines: ['smithi038', 'smithi164']
2023-05-22T10:30:25.271 INFO:teuthology.kill:2023-05-22 10:30:25,271.271 INFO:teuthology.nuke:targets:
2023-05-22T10:30:25.272 INFO:teuthology.kill:  smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIEBFB+cpSJ7+NuuLyoqFy5GzuoZ3GO16wcE7zYEW8hF56tRhrobtIntpKLgtLUfMzEpZR1pzdXKIKk81cT4V1e8r1mBYmR2NTWgTpsd2okSIHeY3dWAai5PgXv/ZKnRmlQQVL+T9WCh5AC7H4I81XpT3Rm5fye+c8n9gUEIIJHcioayn4JuitzS1WzekWyn/TVnQbrW0ugK6TjatLOeHdfV67DOlRLij8yyQ5/+6P6TzQWrVW/naBa4lFvlpr3qxFFGxCKCI+ppd25p3W1AfqrKVtForIJcsrYIKYWFaplUHgHGTRrmzsuk6jGpjwnKtwmull+fnAiL7b1Rb1yCjxZ7L55wBbtz+bbd6b6hanTZqd5aX494k+buNQODJMe03MfUh1ehEb+HaXc4e8eoGRWbWg8FcK07L6jmCpiBpNCMRAdMbV32MblFmTg6WGPsCO8+6kWhZ4A7GxZQKAvaFVJ/iOUvgfKdCNvkfMv6cuSVGK0JVDpqGQN/d9CbSDu8E=
2023-05-22T10:30:25.273 INFO:teuthology.kill:  smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsuTFLtNsOtwdINQpPTeO7Rrv0ULIEDWcEuGTWCr/bbKZRZnPVe8po2IAB/OCbRUuNc6ujMVISWwQKEZMiu4T4=
2023-05-22T10:30:25.273 INFO:teuthology.kill:2023-05-22 10:30:25,271.271 INFO:teuthology.nuke:Checking targets against current locks
2023-05-22T10:30:25.329 INFO:teuthology.kill:2023-05-22 10:30:25,328.328 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-05-22T10:30:25.345 INFO:teuthology.kill:2023-05-22 10:30:25,345.345 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-05-22T10:30:25.403 INFO:teuthology.kill:2023-05-22 10:30:25,403.403 INFO:teuthology.orchestra.console:Power off smithi038
2023-05-22T10:30:25.416 INFO:teuthology.kill:2023-05-22 10:30:25,416.416 INFO:teuthology.orchestra.console:Power off smithi164
2023-05-22T10:30:33.770 INFO:teuthology.kill:2023-05-22 10:30:33,770.770 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2023-05-22T10:30:33.777 INFO:teuthology.kill:2023-05-22 10:30:33,776.776 INFO:teuthology.orchestra.console:Power off for smithi164 completed
2023-05-22T10:30:33.930 INFO:teuthology.kill:2023-05-22 10:30:33,929.929 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com
2023-05-22T10:30:33.938 INFO:teuthology.kill:2023-05-22 10:30:33,938.938 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com
2023-05-22T10:32:34.434 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-05-22T10:32:34.701 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi038 but it was locked by another job: /home/teuthworker/archive/adking-2023-05-22_01:46:07-orch:cephadm-wip-adk2-testing-2023-05-21-1627-reef-distro-default-smithi/7282314
2023-05-22T10:32:34.702 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi164 but it was locked by another job: /home/teuthworker/archive/adking-2023-05-22_01:46:07-orch:cephadm-wip-adk2-testing-2023-05-21-1627-reef-distro-default-smithi/7282314