2022-09-03T12:21:47.513 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18
2022-09-03T12:21:47.743 INFO:teuthology.lock.ops:Start node 'smithi008.front.sepia.ceph.com' reimaging
2022-09-03T12:21:47.745 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: reset os type and version on server
2022-09-03T12:21:47.745 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2022-09-03T12:21:47.768 INFO:teuthology.lock.ops:Node 'smithi008.front.sepia.ceph.com' reimaging is complete
2022-09-03T12:21:47.769 INFO:teuthology.lock.ops:Start node 'smithi157.front.sepia.ceph.com' reimaging
2022-09-03T12:21:47.769 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: reset os type and version on server
2022-09-03T12:21:47.769 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2022-09-03T12:21:47.791 INFO:teuthology.lock.ops:Node 'smithi157.front.sepia.ceph.com' reimaging is complete
2022-09-03T12:21:47.792 INFO:teuthology.lock.ops:Start node 'smithi203.front.sepia.ceph.com' reimaging
2022-09-03T12:21:47.793 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: reset os type and version on server
2022-09-03T12:21:47.793 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2022-09-03T12:21:47.821 INFO:teuthology.lock.ops:Node 'smithi203.front.sepia.ceph.com' reimaging is complete
2022-09-03T12:21:48.004 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of ubuntu 20.04
2022-09-03T12:21:48.011 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of ubuntu 20.04
2022-09-03T12:21:48.041 INFO:teuthology.provision.fog.smithi203:Scheduling deploy of ubuntu 20.04
2022-09-03T12:21:48.358 INFO:teuthology.orchestra.console:Power off smithi008
2022-09-03T12:21:48.376 INFO:teuthology.orchestra.console:Power off smithi157
2022-09-03T12:21:48.388 INFO:teuthology.orchestra.console:Power off smithi203
2022-09-03T12:21:56.748 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2022-09-03T12:21:56.752 INFO:teuthology.orchestra.console:Power off for smithi157 completed
2022-09-03T12:21:56.759 INFO:teuthology.orchestra.console:Power off for smithi203 completed
2022-09-03T12:21:56.850 INFO:teuthology.orchestra.console:Power on smithi008
2022-09-03T12:21:56.857 INFO:teuthology.orchestra.console:Power on smithi157
2022-09-03T12:21:56.864 INFO:teuthology.orchestra.console:Power on smithi203
2022-09-03T12:22:05.237 INFO:teuthology.orchestra.console:Power on for smithi008 completed
2022-09-03T12:22:05.243 INFO:teuthology.orchestra.console:Power on for smithi157 completed
2022-09-03T12:22:05.338 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish
2022-09-03T12:22:05.344 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish
2022-09-03T12:22:05.371 INFO:teuthology.orchestra.console:Power on for smithi203 completed
2022-09-03T12:22:05.473 INFO:teuthology.provision.fog.smithi203:Waiting for deploy to finish
2022-09-03T12:26:57.986 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-03T12:26:58.533 INFO:teuthology.provision.fog.smithi008:Node is ready
2022-09-03T12:26:58.540 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com
2022-09-03T12:26:58.549 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008
2022-09-03T12:26:58.719 INFO:teuthology.provision.fog.smithi008:Deploy complete!
2022-09-03T12:27:18.821 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-03T12:27:19.392 INFO:teuthology.provision.fog.smithi203:Node is ready
2022-09-03T12:27:19.399 INFO:teuthology.orchestra.run.smithi203.stdout:smithi203.front.sepia.ceph.com
2022-09-03T12:27:19.455 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.15.203 smithi203.front.sepia.ceph.com smithi203
2022-09-03T12:27:19.656 INFO:teuthology.provision.fog.smithi203:Deploy complete!
2022-09-03T12:27:35.854 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-09-03T12:27:36.336 INFO:teuthology.provision.fog.smithi157:Node is ready
2022-09-03T12:27:36.383 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com
2022-09-03T12:27:36.391 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157
2022-09-03T12:27:36.538 INFO:teuthology.provision.fog.smithi157:Deploy complete!
2022-09-03T12:27:36.892 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com
2022-09-03T12:27:36.893 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-03T12:27:36.915 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com
2022-09-03T12:27:36.916 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-03T12:27:36.935 INFO:teuthology.lock.ops:Checking smithi203.front.sepia.ceph.com
2022-09-03T12:27:36.936 INFO:teuthology.lock.ops:New key found. Updating...
2022-09-03T12:27:36.961 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server
2022-09-03T12:27:36.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2022-09-03T12:27:37.314 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64
2022-09-03T12:27:37.322 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Ubuntu"
2022-09-03T12:27:37.322 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="20.04.5 LTS (Focal Fossa)"
2022-09-03T12:27:37.323 INFO:teuthology.orchestra.run.smithi008.stdout:ID=ubuntu
2022-09-03T12:27:37.323 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE=debian
2022-09-03T12:27:37.323 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS"
2022-09-03T12:27:37.324 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="20.04"
2022-09-03T12:27:37.324 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.ubuntu.com/"
2022-09-03T12:27:37.324 INFO:teuthology.orchestra.run.smithi008.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-09-03T12:27:37.324 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-09-03T12:27:37.325 INFO:teuthology.orchestra.run.smithi008.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-09-03T12:27:37.325 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_CODENAME=focal
2022-09-03T12:27:37.325 INFO:teuthology.orchestra.run.smithi008.stdout:UBUNTU_CODENAME=focal
2022-09-03T12:27:37.326 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2022-09-03T12:27:37.347 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server
2022-09-03T12:27:37.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com'
2022-09-03T12:27:37.615 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64
2022-09-03T12:27:37.622 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Ubuntu"
2022-09-03T12:27:37.622 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="20.04.5 LTS (Focal Fossa)"
2022-09-03T12:27:37.622 INFO:teuthology.orchestra.run.smithi157.stdout:ID=ubuntu
2022-09-03T12:27:37.623 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE=debian
2022-09-03T12:27:37.623 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS"
2022-09-03T12:27:37.623 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="20.04"
2022-09-03T12:27:37.623 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.ubuntu.com/"
2022-09-03T12:27:37.624 INFO:teuthology.orchestra.run.smithi157.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-09-03T12:27:37.624 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-09-03T12:27:37.624 INFO:teuthology.orchestra.run.smithi157.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-09-03T12:27:37.624 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_CODENAME=focal
2022-09-03T12:27:37.625 INFO:teuthology.orchestra.run.smithi157.stdout:UBUNTU_CODENAME=focal
2022-09-03T12:27:37.625 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server
2022-09-03T12:27:37.648 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: set os type and version on server
2022-09-03T12:27:37.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com'
2022-09-03T12:27:37.872 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64
2022-09-03T12:27:38.013 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Ubuntu"
2022-09-03T12:27:38.014 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="20.04.5 LTS (Focal Fossa)"
2022-09-03T12:27:38.014 INFO:teuthology.orchestra.run.smithi203.stdout:ID=ubuntu
2022-09-03T12:27:38.014 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE=debian
2022-09-03T12:27:38.015 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS"
2022-09-03T12:27:38.015 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="20.04"
2022-09-03T12:27:38.015 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.ubuntu.com/"
2022-09-03T12:27:38.016 INFO:teuthology.orchestra.run.smithi203.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-09-03T12:27:38.016 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-09-03T12:27:38.016 INFO:teuthology.orchestra.run.smithi203.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-09-03T12:27:38.016 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_CODENAME=focal
2022-09-03T12:27:38.017 INFO:teuthology.orchestra.run.smithi203.stdout:UBUNTU_CODENAME=focal
2022-09-03T12:27:38.017 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2022-09-03T12:27:38.141 INFO:teuthology.dispatcher.supervisor:Running job 7004329
2022-09-03T12:27:38.143 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004329 --name benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004329/orig.config.yaml
2022-09-03T12:27:38.147 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-09-01_10:32:47-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004329
2022-09-03T12:27:38.148 INFO:teuthology.dispatcher.supervisor:Job PID: 2344
2022-09-03T12:27:38.149 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-09-04T00:28:01.567 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-09-04T00:28:01.736 INFO:teuthology.kill:Killing Pids: {2344}
2022-09-04T00:28:03.204 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-09-04T00:28:03.205 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-09-04T00:28:03.205 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-09-04T00:28:03.206 INFO:teuthology.misc:Compressing logs...
2022-09-04T00:28:03.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2022-09-04T00:28:03.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com'
2022-09-04T00:28:03.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com'
2022-09-04T00:28:04.125 INFO:teuthology.misc:Compressing logs...
2022-09-04T00:29:45.245 INFO:teuthology.misc:Compressing logs...
2022-09-04T00:29:49.792 INFO:teuthology.kill:No teuthology processes running
2022-09-04T00:29:51.217 INFO:teuthology.kill:Nuking machines: ['smithi008', 'smithi157', 'smithi203']
2022-09-04T00:29:52.375 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-09-04T00:29:52.376 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-09-04T00:29:52.377 INFO:teuthology.kill:2022-09-04 00:29:52,375.375 INFO:teuthology.nuke:targets:
2022-09-04T00:29:52.377 INFO:teuthology.kill:  smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgHXG6sTNSbR7Kahw3W462/GctjY9CWuSZmwLWuHfY1Ben5+lHHGN5Qp15Hn1le1KVjNta3/j3OocsSXlVDFfOyHLAk0bowzV6y5ju/2CJLl2I7Cy3NgGiPwt6GIRbMw+hqjMlD1l1ZyPEFbBgthnKpwYoU1C4itGkqopYd7SR5XxnxhMbwCzzhfjmcS4kf5Blrz0ajxvTyTOuEytBbjrEPM27LjWU3sPeAJWko+KTHL9rfPwBvolWBcndFDOhEytoWouvtZCa6xV4N3AI0JHNQTdIqxcfVZ95suei5FGdMdoI2R995kjsdVu7qRMEKWo3KdG8/R3/K4e0GiydNCi3+xaASkcoU5adfT9Cp9ROLwdWP/X5pQFd8qC6dbcmpI9iQjEmXALIzFNUAyrFo1RI3e4Yo3iwey7ZlSpBpDqm3bXpipUG9Ro/5q/XkzFmW/CxjIvUqpcwTFpHB2Y2sf5L3lLt9+ZM8m4PefI9SqjGn+NNSU0wJvJG6e/9PITzGEU=
2022-09-04T00:29:52.377 INFO:teuthology.kill:  smithi157.front.sepia.ceph.com: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFqmKXWldKIbi7vFGAEG/aQfmoZWglZPWo/0p8GvZYWW
2022-09-04T00:29:52.378 INFO:teuthology.kill:  smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCOw7xiM4rM+gCdPrUney4PBW92YfwbTVQ8jGTE9GltZ4Q09sTg+msgIhLb2jL1yf2JsZxyhszqqKH+ha2LcBqU=
2022-09-04T00:29:52.378 INFO:teuthology.kill:2022-09-04 00:29:52,375.375 INFO:teuthology.nuke:Checking targets against current locks
2022-09-04T00:29:52.442 INFO:teuthology.kill:2022-09-04 00:29:52,442.442 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-04T00:29:52.466 INFO:teuthology.kill:2022-09-04 00:29:52,466.466 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-04T00:29:52.501 INFO:teuthology.kill:2022-09-04 00:29:52,500.500 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-09-04T00:29:52.540 INFO:teuthology.kill:2022-09-04 00:29:52,540.540 INFO:teuthology.orchestra.console:Power off smithi008
2022-09-04T00:29:52.588 INFO:teuthology.kill:2022-09-04 00:29:52,588.588 INFO:teuthology.orchestra.console:Power off smithi157
2022-09-04T00:29:52.628 INFO:teuthology.kill:2022-09-04 00:29:52,628.628 INFO:teuthology.orchestra.console:Power off smithi203
2022-09-04T00:30:01.000 INFO:teuthology.kill:2022-09-04 00:30:01,000.000 INFO:teuthology.orchestra.console:Power off for smithi203 completed
2022-09-04T00:30:01.152 INFO:teuthology.kill:2022-09-04 00:30:01,151.151 INFO:teuthology.lock.ops:unlocked: smithi203.front.sepia.ceph.com
2022-09-04T00:30:05.036 INFO:teuthology.kill:2022-09-04 00:30:05,035.035 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2022-09-04T00:30:05.069 INFO:teuthology.kill:2022-09-04 00:30:05,069.069 INFO:teuthology.orchestra.console:Power off for smithi157 completed
2022-09-04T00:30:05.193 INFO:teuthology.kill:2022-09-04 00:30:05,192.192 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com
2022-09-04T00:30:05.227 INFO:teuthology.kill:2022-09-04 00:30:05,226.226 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com
2022-09-04T00:32:05.582 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-09-04T00:32:05.732 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi008 but it was locked by another job: /home/teuthworker/archive/adking-2022-09-03_23:13:39-orch:cephadm-wip-adk3-testing-2022-09-03-1327-quincy-distro-default-smithi/7009721
2022-09-04T00:32:05.732 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi157 but it was locked by another job: /home/teuthworker/archive/adking-2022-09-03_23:13:39-orch:cephadm-wip-adk3-testing-2022-09-03-1327-quincy-distro-default-smithi/7009719
2022-09-04T00:32:05.733 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi203 but it was locked by another job: /home/teuthworker/archive/adking-2022-09-03_23:13:39-orch:cephadm-wip-adk3-testing-2022-09-03-1327-quincy-distro-default-smithi/7009718