2022-09-04T01:38:27.454 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-04T01:38:27.638 INFO:teuthology.lock.ops:Start node 'smithi037.front.sepia.ceph.com' reimaging 2022-09-04T01:38:27.639 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: reset os type and version on server 2022-09-04T01:38:27.639 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-09-04T01:38:27.670 INFO:teuthology.lock.ops:Node 'smithi037.front.sepia.ceph.com' reimaging is complete 2022-09-04T01:38:27.671 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2022-09-04T01:38:27.671 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2022-09-04T01:38:27.672 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-09-04T01:38:27.694 INFO:teuthology.lock.ops:Node 'smithi159.front.sepia.ceph.com' reimaging is complete 2022-09-04T01:38:27.891 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of rhel 8.6 2022-09-04T01:38:27.917 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of rhel 8.6 2022-09-04T01:38:28.243 INFO:teuthology.orchestra.console:Power off smithi037 2022-09-04T01:38:28.269 INFO:teuthology.orchestra.console:Power off smithi159 2022-09-04T01:38:28.416 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-09-04T01:38:28.518 INFO:teuthology.orchestra.console:Power on smithi159 2022-09-04T01:38:32.793 INFO:teuthology.orchestra.console:Power on for smithi159 completed 2022-09-04T01:38:32.894 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2022-09-04T01:38:36.632 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-09-04T01:38:36.733 INFO:teuthology.orchestra.console:Power on smithi037 2022-09-04T01:38:45.120 INFO:teuthology.orchestra.console:Power on for smithi037 completed 2022-09-04T01:38:45.222 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2022-09-04T01:42:52.928 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-04T01:42:53.174 INFO:teuthology.provision.fog.smithi037:Node is ready 2022-09-04T01:42:53.194 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2022-09-04T01:42:53.254 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2022-09-04T01:42:53.647 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2022-09-04T01:42:57.254 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-04T01:43:17.534 INFO:teuthology.provision.fog.smithi159:Node is ready 2022-09-04T01:43:17.557 INFO:teuthology.orchestra.run.smithi159.stdout:smithi055 2022-09-04T01:43:17.780 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2022-09-04T01:43:18.050 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2022-09-04T01:43:18.051 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-04T01:43:18.081 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2022-09-04T01:43:18.082 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-04T01:43:18.107 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2022-09-04T01:43:18.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-09-04T01:43:18.430 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2022-09-04T01:43:18.500 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="Red Hat Enterprise Linux" 2022-09-04T01:43:18.501 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8.6 (Ootpa)" 2022-09-04T01:43:18.502 INFO:teuthology.orchestra.run.smithi037.stdout:ID="rhel" 2022-09-04T01:43:18.502 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="fedora" 2022-09-04T01:43:18.503 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8.6" 2022-09-04T01:43:18.504 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2022-09-04T01:43:18.504 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-04T01:43:18.504 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2022-09-04T01:43:18.505 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-04T01:43:18.505 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://www.redhat.com/" 2022-09-04T01:43:18.506 INFO:teuthology.orchestra.run.smithi037.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-04T01:43:18.506 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-04T01:43:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: 2022-09-04T01:43:18.507 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-04T01:43:18.507 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-04T01:43:18.508 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-04T01:43:18.508 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-04T01:43:18.509 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2022-09-04T01:43:18.532 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2022-09-04T01:43:18.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-09-04T01:43:18.748 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2022-09-04T01:43:18.810 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux" 2022-09-04T01:43:18.810 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.6 (Ootpa)" 2022-09-04T01:43:18.811 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel" 2022-09-04T01:43:18.812 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora" 2022-09-04T01:43:18.812 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.6" 2022-09-04T01:43:18.813 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2022-09-04T01:43:18.813 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-04T01:43:18.813 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2022-09-04T01:43:18.817 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-04T01:43:18.817 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/" 2022-09-04T01:43:18.818 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-04T01:43:18.818 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-04T01:43:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-09-04T01:43:18.819 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-04T01:43:18.820 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-04T01:43:18.820 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-04T01:43:18.820 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-04T01:43:18.823 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-09-04T01:43:19.010 INFO:teuthology.dispatcher.supervisor:Running job 7004480 2022-09-04T01:43:19.011 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004480 --name benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} -- /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004480/orig.config.yaml 2022-09-04T01:43:19.025 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004480 2022-09-04T01:43:19.027 INFO:teuthology.dispatcher.supervisor:Job PID: 20557 2022-09-04T01:43:19.028 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-04T13:44:17.368 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-04T13:44:17.521 INFO:teuthology.kill:Killing Pids: {20557} 2022-09-04T13:44:19.082 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-09-04T13:44:19.083 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-09-04T13:44:19.083 INFO:teuthology.misc:Compressing logs... 2022-09-04T13:44:19.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2022-09-04T13:44:19.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-09-04T13:44:19.613 INFO:teuthology.misc:Compressing logs... 2022-09-04T13:44:20.620 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-04T13:44:20.831 INFO:teuthology.orchestra.run.smithi159.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-09-04T13:44:37.471 INFO:teuthology.misc:Compressing logs... 2022-09-04T13:44:42.464 INFO:teuthology.kill:No teuthology processes running 2022-09-04T13:44:43.866 INFO:teuthology.kill:Nuking machines: ['smithi037', 'smithi159'] 2022-09-04T13:44:45.063 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-04T13:44:45.064 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-04T13:44:45.065 INFO:teuthology.kill:2022-09-04 13:44:45,063.063 INFO:teuthology.nuke:targets: 2022-09-04T13:44:45.065 INFO:teuthology.kill: smithi037.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+dLUin//nkx0QpjiBCGyjkG1fnz72Px4Czqr1dBSiq6p88Bg6p0OuwPLxWvd1TiC0FVJY1gatYX+xXMAHqtuWmKz9zID34VBBhQDmAdA30G4mTe+AgbXsHEGTxyygev/M1CxDUS2gi4g9dvjXAdiqpUV3moM4u6yDc1Lm7dxnRPyXQxH7jdhNUbb3mOXq0DdVnbJ9PIxotf2Fo8t/XV5qBdxdhEc8lmkgz5tE21+9M1aVU5YLd4vIGtyYdt8y7EM4DHWCemWqZecKOOuVg8HYmdnby4Vtk1GYsW0t5fL5W1aTUncDoL1xJKKLd1OPWKwaqhzoqMmaOcHBVKXno8hZCEG9sdmJSHTzGb4JO95uwaOUV0X2Pz1pNjOjalhaBfSJjYNS+BIBNlKrva6CyOtge7kiJp2eKx5zwPA9yekf7c8hL6DHXv4yOdXnqn6uJnqfx0ELZjajGv3uo0nIdXl53YdOc9zEbVZwWFR6CrgPOskyto48odixFlkxVVlQM08= 2022-09-04T13:44:45.066 INFO:teuthology.kill: smithi159.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDDxOXNNKOGdWgg4lq42zJR7V/bgSr9YposRYhFplIRuwpqNOdu7fHqhKsSilJZy/AYU3KGAWw4MyFpP1N9MUDCBBccXz9XLVQHL7uXDLE7V0YWSmmVO7pHaJHiwZ5fufm5CEzTvk/vzagBKlsW72IzSYjyY0avwRsr8U/FeyjMfKKtelk25zatfHScMRs653e3mRDj0rup88ma+GN9EqW2ofic7F/rwff0YLTFBynjVWAv9o0Fvj5CvSHf9NfT69WgGRQbLiA/11oIdJncOwW9luvW8u/V/XMSUkxTvmzkLmdilnxsoSNl8Vqu0icNTOGsUAc/PyYHJKi98GwiHqZwFVxUk12cSCjglqiMDil8GUn39oL3rLZoN/pK5gedl0GCNXxCcVn1CIsCXGGu6opNKBKrN4DqfZHiHg3BjT49wjaZSJ0Cy7FQ2wd3Lj74jvv5R9tq5CZmlTHKc2cXdXBnVBalLkS6SZz4ACcYpr2sDTXWGc6YIvqZtPQyyZ7Tx8= 2022-09-04T13:44:45.066 INFO:teuthology.kill:2022-09-04 13:44:45,063.063 INFO:teuthology.nuke:Checking targets against current locks 2022-09-04T13:44:45.141 INFO:teuthology.kill:2022-09-04 13:44:45,141.141 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-04T13:44:45.187 INFO:teuthology.kill:2022-09-04 13:44:45,187.187 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-04T13:44:45.260 INFO:teuthology.kill:2022-09-04 13:44:45,260.260 INFO:teuthology.orchestra.console:Power off smithi037 2022-09-04T13:44:45.310 INFO:teuthology.kill:2022-09-04 13:44:45,310.310 INFO:teuthology.orchestra.console:Power off smithi159 2022-09-04T13:44:53.645 INFO:teuthology.kill:2022-09-04 13:44:53,644.644 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2022-09-04T13:44:53.798 INFO:teuthology.kill:2022-09-04 13:44:53,797.797 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com 2022-09-04T13:44:57.787 INFO:teuthology.kill:2022-09-04 13:44:57,787.787 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-09-04T13:44:57.943 INFO:teuthology.kill:2022-09-04 13:44:57,943.943 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com 2022-09-04T13:46:58.308 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-04T13:46:58.415 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi037 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-09-01_10:54:32-rados-WIP_GBH_snap_mapper_one_way-distro-default-smithi/7004638