2023-03-25T10:39:32.106 INFO:root:teuthology version: 0.0.1.dev1+ge372916
2023-03-25T10:39:32.265 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging
2023-03-25T10:39:32.267 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server
2023-03-25T10:39:32.267 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-03-25T10:39:32.291 INFO:teuthology.lock.ops:Node 'smithi023.front.sepia.ceph.com' reimaging is complete
2023-03-25T10:39:32.291 INFO:teuthology.lock.ops:Start node 'smithi142.front.sepia.ceph.com' reimaging
2023-03-25T10:39:32.291 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: reset os type and version on server
2023-03-25T10:39:32.292 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server
2023-03-25T10:39:32.316 INFO:teuthology.lock.ops:Node 'smithi142.front.sepia.ceph.com' reimaging is complete
2023-03-25T10:39:32.463 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of centos 8.stream
2023-03-25T10:39:32.466 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 8.stream
2023-03-25T10:39:32.671 INFO:teuthology.orchestra.console:Power off smithi023
2023-03-25T10:39:32.687 INFO:teuthology.orchestra.console:Power off smithi142
2023-03-25T10:39:32.822 INFO:teuthology.orchestra.console:Power off for smithi023 completed
2023-03-25T10:39:32.833 INFO:teuthology.orchestra.console:Power off for smithi142 completed
2023-03-25T10:39:32.924 INFO:teuthology.orchestra.console:Power on smithi023
2023-03-25T10:39:32.934 INFO:teuthology.orchestra.console:Power on smithi142
2023-03-25T10:39:41.305 INFO:teuthology.orchestra.console:Power on for smithi142 completed
2023-03-25T10:39:41.318 INFO:teuthology.orchestra.console:Power on for smithi023 completed
2023-03-25T10:39:41.406 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish
2023-03-25T10:39:41.419 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish
2023-03-25T10:42:30.345 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142
2023-03-25T10:42:39.429 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142
2023-03-25T10:42:48.485 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142
2023-03-25T10:42:57.545 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142
2023-03-25T10:42:57.599 ERROR:teuthology.orchestra.connection:Error authenticating with smithi023.front.sepia.ceph.com: Authentication failed.
2023-03-25T10:42:58.717 ERROR:teuthology.orchestra.connection:Error authenticating with smithi023.front.sepia.ceph.com: Authentication failed.
2023-03-25T10:42:59.834 ERROR:teuthology.orchestra.connection:Error authenticating with smithi023.front.sepia.ceph.com: Authentication failed.
2023-03-25T10:43:00.953 ERROR:teuthology.orchestra.connection:Error authenticating with smithi023.front.sepia.ceph.com: Authentication failed.
2023-03-25T10:43:06.597 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142
2023-03-25T10:43:15.653 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.142
2023-03-25T10:43:53.482 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-25T10:44:01.954 WARNING:teuthology.provision.fog:timed out
2023-03-25T10:44:11.013 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-03-25T10:44:13.680 INFO:teuthology.provision.fog.smithi142:Node is ready
2023-03-25T10:44:13.700 INFO:teuthology.orchestra.run.smithi142.stdout:smithi001
2023-03-25T10:44:13.757 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.1 smithi001 smithi001
2023-03-25T10:44:14.127 INFO:teuthology.provision.fog.smithi142:Deploy complete!
2023-03-25T10:44:20.069 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-03-25T10:44:29.125 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-03-25T10:45:35.128 WARNING:teuthology.provision.fog:timed out
2023-03-25T10:45:41.422 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-25T10:45:41.701 INFO:teuthology.provision.fog.smithi023:Node is ready
2023-03-25T10:45:41.721 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com
2023-03-25T10:45:41.781 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023
2023-03-25T10:45:42.138 INFO:teuthology.provision.fog.smithi023:Deploy complete!
2023-03-25T10:45:42.310 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com
2023-03-25T10:45:42.312 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-25T10:45:42.333 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com
2023-03-25T10:45:42.333 INFO:teuthology.lock.ops:New key found. Updating...
2023-03-25T10:45:42.359 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server
2023-03-25T10:45:42.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-03-25T10:45:42.507 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64
2023-03-25T10:45:42.565 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream"
2023-03-25T10:45:42.565 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8"
2023-03-25T10:45:42.565 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2023-03-25T10:45:42.565 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2023-03-25T10:45:42.565 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-25T10:45:42.566 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-25T10:45:42.567 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-03-25T10:45:42.591 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server
2023-03-25T10:45:42.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com'
2023-03-25T10:45:42.788 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64
2023-03-25T10:45:42.847 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream"
2023-03-25T10:45:42.848 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8"
2023-03-25T10:45:42.848 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos"
2023-03-25T10:45:42.848 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora"
2023-03-25T10:45:42.848 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8"
2023-03-25T10:45:42.848 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8"
2023-03-25T10:45:42.848 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 8"
2023-03-25T10:45:42.848 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31"
2023-03-25T10:45:42.849 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-03-25T10:45:42.849 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/"
2023-03-25T10:45:42.849 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-03-25T10:45:42.849 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-03-25T10:45:42.849 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-03-25T10:45:42.850 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server
2023-03-25T10:45:42.965 INFO:teuthology.dispatcher.supervisor:Running job 7211075
2023-03-25T10:45:42.966 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_main/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi/7211075 --name benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} -- /home/teuthworker/archive/benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi/7211075/orig.config.yaml
2023-03-25T10:45:42.970 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2023-03-16_20:57:19-rados-WIP_GBH_snap_mapper_5-distro-default-smithi/7211075
2023-03-25T10:45:42.971 INFO:teuthology.dispatcher.supervisor:Job PID: 25388
2023-03-25T10:45:42.972 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-03-25T22:46:06.818 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2023-03-25T22:46:06.963 INFO:teuthology.kill:Killing Pids: {25388}
2023-03-25T22:46:07.834 INFO:teuthology.task.internal:roles: ubuntu@smithi023.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2023-03-25T22:46:07.835 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2023-03-25T22:46:07.835 INFO:teuthology.misc:Compressing logs...
2023-03-25T22:46:07.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-03-25T22:46:07.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com'
2023-03-25T22:46:08.361 INFO:teuthology.misc:Compressing logs...
2023-03-25T22:46:09.276 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-25T22:46:09.318 INFO:teuthology.orchestra.run.smithi023.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2023-03-25T22:46:37.229 INFO:teuthology.misc:Compressing logs...
2023-03-25T22:46:44.236 INFO:teuthology.kill:No teuthology processes running
2023-03-25T22:46:45.046 INFO:teuthology.kill:Nuking machines: ['smithi023', 'smithi142']
2023-03-25T22:46:45.644 INFO:teuthology.kill:2023-03-25 22:46:45,643.643 INFO:teuthology.nuke:targets:
2023-03-25T22:46:45.644 INFO:teuthology.kill:  smithi023.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEj3MmfWngDLX9KPsVtd/LYEwgkQiAWD5oC/8+ssqI8m2IW22s8qkLnbl8nyGwSKez7EGcd6nVKVpxGZSPlKi38=
2023-03-25T22:46:45.645 INFO:teuthology.kill:  smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkJnd3mzjqh0rrgS9ExAyV3m8Dc3Br7rIDIYvTy0+ZlbXp2EQzYxBaucKPRoEVeaYWuRao1UGSXcTjx6mjRZKcvlxlVZjc4aN1MI0T1pdGQsOpfJeov4philtXiQ9a0bs1awL0RzngBzIKkhfs9Q9EyzbZyCkxEvSzMvpLBG68CU21cY3lZ1D/CXAI8otRoWEACUn380EVLk0P6f+lYCM5xzQV2eZTWt00T/udW13wqYchdTM5Ph4tOjlLqFhgH2j1JJVLBk9a73Q2KIpGDpI0ImufKvv/NrkogwLMwW2ZyRvcN6bhDas2ZU2HwrwTcDyhezOu5MhmW5pR7x1wYWrzJ3ga/8Wrq5EOCh6n+IDV0IqevdAmiXRYvzNgjtToxcLoZWbOfm0AA43GWsV2BYOnL36sMBZYz6kEXrfFpv2V6E11bFVM2pPdnKXt8k5X/Nl0XZYfcu7g5TcB6yi781aZe2b4ub8cCnVPe5w6uUmS85lpaM1/vIljHDcH/BlLx4c=
2023-03-25T22:46:45.645 INFO:teuthology.kill:2023-03-25 22:46:45,643.643 INFO:teuthology.nuke:Checking targets against current locks
2023-03-25T22:46:45.703 INFO:teuthology.kill:2023-03-25 22:46:45,703.703 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-25T22:46:45.731 INFO:teuthology.kill:2023-03-25 22:46:45,731.731 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-25T22:46:45.785 INFO:teuthology.kill:2023-03-25 22:46:45,784.784 INFO:teuthology.orchestra.console:Power off smithi023
2023-03-25T22:46:45.802 INFO:teuthology.kill:2023-03-25 22:46:45,802.802 INFO:teuthology.orchestra.console:Power off smithi142
2023-03-25T22:46:54.166 INFO:teuthology.kill:2023-03-25 22:46:54,166.166 INFO:teuthology.orchestra.console:Power off for smithi142 completed
2023-03-25T22:46:54.328 INFO:teuthology.kill:2023-03-25 22:46:54,327.327 INFO:teuthology.lock.ops:unlocked: smithi142.front.sepia.ceph.com
2023-03-25T22:46:58.270 INFO:teuthology.kill:2023-03-25 22:46:58,270.270 INFO:teuthology.orchestra.console:Power off for smithi023 completed
2023-03-25T22:46:58.425 INFO:teuthology.kill:2023-03-25 22:46:58,425.425 INFO:teuthology.lock.ops:unlocked: smithi023.front.sepia.ceph.com
2023-03-25T22:48:58.684 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2023-03-25T22:48:58.795 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi023 but it was locked by another job: /home/teuthworker/archive/bhubbard-2023-03-20_00:19:12-powercycle-main-distro-default-smithi/7213817
2023-03-25T22:48:58.796 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi142 but it was locked by another job: /home/teuthworker/archive/bhubbard-2023-03-20_00:19:12-powercycle-main-distro-default-smithi/7213817