2022-06-28T06:44:01.739 INFO:root:teuthology version: 1.1.1.dev376+g082ae7ef 2022-06-28T06:44:01.886 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging 2022-06-28T06:44:01.888 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server 2022-06-28T06:44:01.888 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2022-06-28T06:44:01.923 INFO:teuthology.lock.ops:Node 'smithi119.front.sepia.ceph.com' reimaging is complete 2022-06-28T06:44:01.923 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging 2022-06-28T06:44:01.923 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server 2022-06-28T06:44:01.924 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-06-28T06:44:01.957 INFO:teuthology.lock.ops:Node 'smithi153.front.sepia.ceph.com' reimaging is complete 2022-06-28T06:44:02.156 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of centos 8.stream 2022-06-28T06:44:02.160 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 8.stream 2022-06-28T06:44:02.538 INFO:teuthology.orchestra.console:Power off smithi119 2022-06-28T06:44:02.580 INFO:teuthology.orchestra.console:Power off smithi153 2022-06-28T06:44:10.953 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-06-28T06:44:11.055 INFO:teuthology.orchestra.console:Power on smithi153 2022-06-28T06:44:15.041 INFO:teuthology.orchestra.console:Power off for smithi119 completed 2022-06-28T06:44:15.149 INFO:teuthology.orchestra.console:Power on smithi119 2022-06-28T06:44:19.405 INFO:teuthology.orchestra.console:Power on for smithi119 completed 2022-06-28T06:44:19.449 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2022-06-28T06:44:19.507 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish 2022-06-28T06:44:19.550 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2022-06-28T06:48:51.384 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-28T06:48:51.710 INFO:teuthology.provision.fog.smithi119:Node is ready 2022-06-28T06:48:51.730 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119.front.sepia.ceph.com 2022-06-28T06:48:51.791 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119 smithi119.front.sepia.ceph.com smithi119 2022-06-28T06:48:52.179 INFO:teuthology.provision.fog.smithi119:Deploy complete! 2022-06-28T06:49:06.410 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-28T06:49:06.708 INFO:teuthology.provision.fog.smithi153:Node is ready 2022-06-28T06:49:06.726 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2022-06-28T06:49:06.788 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2022-06-28T06:49:07.172 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2022-06-28T06:49:07.378 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com 2022-06-28T06:49:07.380 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-28T06:49:07.411 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2022-06-28T06:49:07.413 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-28T06:49:07.434 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server 2022-06-28T06:49:07.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2022-06-28T06:49:07.787 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2022-06-28T06:49:07.845 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2022-06-28T06:49:07.845 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8" 2022-06-28T06:49:07.845 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2022-06-28T06:49:07.846 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2022-06-28T06:49:07.846 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8" 2022-06-28T06:49:07.846 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2022-06-28T06:49:07.847 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-28T06:49:07.847 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2022-06-28T06:49:07.847 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-28T06:49:07.848 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2022-06-28T06:49:07.848 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-28T06:49:07.848 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-28T06:49:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-28T06:49:07.850 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2022-06-28T06:49:07.911 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2022-06-28T06:49:07.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-06-28T06:49:08.161 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2022-06-28T06:49:08.220 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2022-06-28T06:49:08.220 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8" 2022-06-28T06:49:08.220 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2022-06-28T06:49:08.221 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2022-06-28T06:49:08.221 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8" 2022-06-28T06:49:08.221 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2022-06-28T06:49:08.221 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-28T06:49:08.222 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2022-06-28T06:49:08.222 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-28T06:49:08.222 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2022-06-28T06:49:08.222 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-28T06:49:08.223 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-28T06:49:08.223 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-28T06:49:08.224 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-06-28T06:49:08.376 INFO:teuthology.dispatcher.supervisor:Running job 6901577 2022-06-28T06:49:08.377 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_082ae7ef4302fa54665ed0a2535e8e254118dcfd/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901577 --name lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901577/orig.config.yaml 2022-06-28T06:49:08.387 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901577 2022-06-28T06:49:08.391 INFO:teuthology.dispatcher.supervisor:Job PID: 6383 2022-06-28T06:49:08.391 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-06-28T18:50:06.647 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-06-28T18:50:06.865 INFO:teuthology.kill:Killing Pids: {6383} 2022-06-28T18:50:08.306 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-06-28T18:50:08.306 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-06-28T18:50:08.307 INFO:teuthology.misc:Compressing logs... 2022-06-28T18:50:08.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2022-06-28T18:50:08.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-06-28T18:50:09.357 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-28T18:50:09.545 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-06-28T18:50:10.150 INFO:teuthology.kill:No teuthology processes running 2022-06-28T18:50:11.490 INFO:teuthology.kill:Nuking machines: ['smithi119', 'smithi153'] 2022-06-28T18:50:12.578 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_082ae7ef4302fa54665ed0a2535e8e254118dcfd/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-06-28T18:50:12.580 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-06-28T18:50:12.580 INFO:teuthology.kill:2022-06-28 18:50:12,578.578 INFO:teuthology.nuke:targets: 2022-06-28T18:50:12.581 INFO:teuthology.kill: smithi119.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC09jYxH28GII7v2TeWK6EqL66YYohvydAT9UEVCsV5GXM9ilviVdUC4kpP8u9GtA2E0zU4e7z2sUMTN8sTMBPbFjj1jn9e8fd8px07NKskJY7qAMa8g904u8358DWbUYUYl2ZHixV7AoHSs0y/JMNORu5Ih9BedY6LopDymfVEmggQIA6nu3X2D76k4Htc9cSLSblAgSjjJg9gwjQhqukA+dFej6APCgMJ4SkdEhh6d0Iczv1HX5yBGDI9Qgtb2rPiJkZ2Ygu4RNcR2nJdxPAAmyk8Q6udesS9iBdOC5KAmWCoOjdmDeEemH58ocmqzWOw3fgcfv5jgRiz1+ZrmzZdTY1/sIomvY7NnRLN4nTsnKVx680v3xQN/qaHKKFV+iSt04blp1PzUGp9Z66kl4ZpLUXhS5GNNTEcPToVxLSuZGFMBDyR3ncvFW5zGxsV4NH7mstKfs6AD8bwyplwKy3uDVMK5KOeWH/wVCl3BlsuyJXHwQ/lqi8eqGp0vGww6ak= 2022-06-28T18:50:12.581 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlbr/jC8u2SVTGIsQM2yDZCcYxtajSi2KvnsahjORGIjTrir3Wlv8o2u27AyXcSl6tbMROrXY2DvzwEZYZGj6k1NWfEGd6vB3a5dfNolDhLxrLA2k4Tqq35f18Yc9Mya7MbqCdPOt6hunqRAiwswmmX/pK1z0ieteEelYHi/kb5CAc7m3l3lTBYSizqNonZHyKiB6BuyMWPId+cYdCkjX/sHdX9t3ifC8GSyQVJpOaIPMNuq+l6yAbgENu8w9XolP3ns4yg4yVSOc4Bzv2V2tCIrcNQNvgJyhY5ydIl0+HLX0Mnk771MAN8o6ndlcJZucVIVjpvGIAwOXchVxy6hTkh7VQSdlKA3uAFCBS6uKUGMDJc2KtDdDE8uZ2jGDWWdPTgXboQTk3t/1S+BW6JzA/ONSdHfPeK6Mfrw6NaH6IFDif930u4VlZVE5O6lQuT+GwTnNJFVvYEs8m+Wbi6pYKyGaLc0D7SkN0rkMJjhzSmbSMpg+a+KMo6qXJ11lIhCE= 2022-06-28T18:50:12.582 INFO:teuthology.kill:2022-06-28 18:50:12,578.578 INFO:teuthology.nuke:Checking targets against current locks 2022-06-28T18:50:12.632 INFO:teuthology.kill:2022-06-28 18:50:12,632.632 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-28T18:50:12.659 INFO:teuthology.kill:2022-06-28 18:50:12,658.658 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-28T18:50:12.714 INFO:teuthology.kill:2022-06-28 18:50:12,714.714 INFO:teuthology.orchestra.console:Power off smithi119 2022-06-28T18:50:12.759 INFO:teuthology.kill:2022-06-28 18:50:12,759.759 INFO:teuthology.orchestra.console:Power off smithi153 2022-06-28T18:50:21.097 INFO:teuthology.kill:2022-06-28 18:50:21,096.096 INFO:teuthology.orchestra.console:Power off for smithi119 completed 2022-06-28T18:50:21.125 INFO:teuthology.kill:2022-06-28 18:50:21,125.125 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-06-28T18:50:21.248 INFO:teuthology.kill:2022-06-28 18:50:21,247.247 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com 2022-06-28T18:50:21.278 INFO:teuthology.kill:2022-06-28 18:50:21,277.277 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com 2022-06-28T18:52:21.617 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-06-28T18:52:21.722 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi119 but it was locked by another job: /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901686 2022-06-28T18:52:21.722 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/lflores-2022-06-27_23:44:12-rados:thrash-erasure-code-wip-yuri2-testing-2022-04-26-1132-octopus-distro-default-smithi/6901686